[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:33:16 dist-test-slave-h3ft krb5kdc[16162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:33:16 dist-test-slave-h3ft krb5kdc[16162](info): set up 2 sockets
Aug 06 22:33:16 dist-test-slave-h3ft krb5kdc[16162](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 22:33:17.736342 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.678s user 0.012s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:33:17 dist-test-slave-h3ft krb5kdc[16162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:33:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:33:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:33:17.997892 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:40781
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:44421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:40781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:18.575608 16185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:18.576449 16185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:18.576658 16185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:18.587879 16185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:33:18.588068 16185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:18.588244 16185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:18.588362 16185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:33:18.588541 16185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:33:18.602536 16185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:44421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:40781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:40781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:18.605402 16185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:18.607877 16185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:18.631920 16194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:18.631969 16192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:18.631899 16191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:18.634498 16185 server_base.cc:1034] running on GCE node
I20260806 22:33:18.636219 16185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:18.638743 16185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:18.642117 16185 hybrid_clock.cc:648] HybridClock initialized: now 1786055598641969 us; error 129 us; skew 500 ppm
I20260806 22:33:18.643043 16185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:18 dist-test-slave-h3ft krb5kdc[16162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055598, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:18.648613 16185 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:33:18.652382 16185 webserver.cc:460] Webserver started at http://127.15.198.190:38127/ using document root <none> and password file <none>
I20260806 22:33:18.653878 16185 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:18.654150 16185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:18.654879 16185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:18.660586 16185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "75f5a60d47de42e1aa90ce7973103e8e"
format_stamp: "Formatted at 2026-08-06 22:33:18 on dist-test-slave-h3ft"
server_key: "ab3062e47bf5a4227b1d14e18c9dc1af"
server_key_iv: "7d038715b69d8162a19ee921f7473260"
server_key_version: "encryptionkey@0"
I20260806 22:33:18.661799 16185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "75f5a60d47de42e1aa90ce7973103e8e"
format_stamp: "Formatted at 2026-08-06 22:33:18 on dist-test-slave-h3ft"
server_key: "ab3062e47bf5a4227b1d14e18c9dc1af"
server_key_iv: "7d038715b69d8162a19ee921f7473260"
server_key_version: "encryptionkey@0"
I20260806 22:33:18.672494 16185 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.011s sys 0.000s
I20260806 22:33:18.680140 16205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:18.682675 16185 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.002s
I20260806 22:33:18.683180 16185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "75f5a60d47de42e1aa90ce7973103e8e"
format_stamp: "Formatted at 2026-08-06 22:33:18 on dist-test-slave-h3ft"
server_key: "ab3062e47bf5a4227b1d14e18c9dc1af"
server_key_iv: "7d038715b69d8162a19ee921f7473260"
server_key_version: "encryptionkey@0"
I20260806 22:33:18.683513 16185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:18.708693 16185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:18.716117 16185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:18.716686 16185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:18.752931 16266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:40781 every 8 connection(s)
I20260806 22:33:18.753026 16185 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:40781
I20260806 22:33:18.755901 16185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:33:18.763221 16268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:18.764456 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 16185
I20260806 22:33:18.764921 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:33:18.767709 16154 external_mini_cluster.cc:1428] Setting key 811a48ce51df8e0851373ecba6b7eb85
I20260806 22:33:18.783983 16268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e: Bootstrap starting.
I20260806 22:33:18.790897 16268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:18.793246 16268 log.cc:824] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:33:18.798555 16268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e: No bootstrap required, opened a new log
I20260806 22:33:18.810302 16268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75f5a60d47de42e1aa90ce7973103e8e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40781 } }
I20260806 22:33:18.811044 16268 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:18.811295 16268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f5a60d47de42e1aa90ce7973103e8e, State: Initialized, Role: FOLLOWER
Aug 06 22:33:18 dist-test-slave-h3ft krb5kdc[16162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:18.812779 16268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75f5a60d47de42e1aa90ce7973103e8e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40781 } }
I20260806 22:33:18.813261 16268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:33:18.813577 16268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:33:18.813903 16268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:18.817821 16268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75f5a60d47de42e1aa90ce7973103e8e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40781 } }
I20260806 22:33:18.819160 16268 leader_election.cc:302] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75f5a60d47de42e1aa90ce7973103e8e; no voters:
I20260806 22:33:18.820403 16268 leader_election.cc:288] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:33:18.821074 16273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:33:18.822721 16273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [term 1 LEADER]: Becoming Leader. State: Replica: 75f5a60d47de42e1aa90ce7973103e8e, State: Running, Role: LEADER
I20260806 22:33:18.824101 16273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75f5a60d47de42e1aa90ce7973103e8e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40781 } }
I20260806 22:33:18.824719 16268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:33:18.833678 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:18.774661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42882 (local address 127.15.198.190:40781)
0806 22:33:18.776496 (+ 1835us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:18.776520 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:18.778208 (+ 1688us) server_negotiation.cc:407] Connection header received
0806 22:33:18.792238 (+ 14030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:18.792275 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:18.793000 (+ 725us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:18.793674 (+ 674us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:18.799295 (+ 5621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:18.801085 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:18.804014 (+ 2929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:18.805310 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:18.805725 (+ 415us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:18.815528 (+ 9803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:18.815554 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:18.815580 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:18.815645 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:18.821967 (+ 6322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:18.823939 (+ 1972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:18.823949 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:18.823979 (+ 30us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:18.824437 (+ 458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:18.825318 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:18.825322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:18.825326 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:18.826660 (+ 1334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:18.828401 (+ 1741us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:18.831203 (+ 2802us) server_negotiation.cc:299] Negotiation successful
0806 22:33:18.831865 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1359,"thread_start_us":903,"threads_started":1}
I20260806 22:33:18.836616 16276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75f5a60d47de42e1aa90ce7973103e8e. Latest consensus state: current_term: 1 leader_uuid: "75f5a60d47de42e1aa90ce7973103e8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75f5a60d47de42e1aa90ce7973103e8e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40781 } } }
I20260806 22:33:18.839205 16276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [sys.catalog]: This master's current role is: LEADER
I20260806 22:33:18.836696 16274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75f5a60d47de42e1aa90ce7973103e8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75f5a60d47de42e1aa90ce7973103e8e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40781 } } }
I20260806 22:33:18.841223 16274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e [sys.catalog]: This master's current role is: LEADER
I20260806 22:33:18.872900 16282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:33:18.888480 16282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:33:18.908658 16282 catalog_manager.cc:1347] Generated new cluster ID: 74cead1248df4f109f9263fa4fb2b4a8
I20260806 22:33:18.908893 16282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:33:18.922437 16282 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:33:18.925019 16282 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:33:18.943768 16282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75f5a60d47de42e1aa90ce7973103e8e: Generated new TSK 0
I20260806 22:33:18.945451 16282 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
I20260806 22:33:19.028038 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:40781
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:44421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:19.612274 16302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:19.612812 16302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:19.613022 16302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:19.622700 16302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:19.622958 16302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:19.623314 16302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:33:19.637727 16302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:44421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:40781
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:19.639731 16302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:19.641495 16302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:19.661823 16314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:19.661690 16312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:19.662067 16317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:19.663795 16302 server_base.cc:1034] running on GCE node
I20260806 22:33:19.664777 16302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:19.666268 16302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:19.667563 16302 hybrid_clock.cc:648] HybridClock initialized: now 1786055599667460 us; error 73 us; skew 500 ppm
I20260806 22:33:19.668111 16302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:19 dist-test-slave-h3ft krb5kdc[16162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055599, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:19.678296 16302 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:33:19.680835 16302 webserver.cc:460] Webserver started at http://127.15.198.129:42421/ using document root <none> and password file <none>
I20260806 22:33:19.681834 16302 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:19.682024 16302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:19.682503 16302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:19.686627 16302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "9dd91e5a816b4960895e3dce74d1cc94"
format_stamp: "Formatted at 2026-08-06 22:33:19 on dist-test-slave-h3ft"
server_key: "3341d0adb8e9d935b7e17e06d11e77d3"
server_key_iv: "3ee1dca08fc2a3dd802960015c729c0e"
server_key_version: "encryptionkey@0"
I20260806 22:33:19.687754 16302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "9dd91e5a816b4960895e3dce74d1cc94"
format_stamp: "Formatted at 2026-08-06 22:33:19 on dist-test-slave-h3ft"
server_key: "3341d0adb8e9d935b7e17e06d11e77d3"
server_key_iv: "3ee1dca08fc2a3dd802960015c729c0e"
server_key_version: "encryptionkey@0"
I20260806 22:33:19.695281 16302 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 22:33:19.700403 16326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:19.702486 16302 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260806 22:33:19.702768 16302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "9dd91e5a816b4960895e3dce74d1cc94"
format_stamp: "Formatted at 2026-08-06 22:33:19 on dist-test-slave-h3ft"
server_key: "3341d0adb8e9d935b7e17e06d11e77d3"
server_key_iv: "3ee1dca08fc2a3dd802960015c729c0e"
server_key_version: "encryptionkey@0"
I20260806 22:33:19.703198 16302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:19.726178 16302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:19.733484 16302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:19.733973 16302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:19.735809 16302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:33:19.738125 16302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:19.738291 16302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:19.738514 16302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:19.738602 16302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:19.808393 16302 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:39167
I20260806 22:33:19.808461 16488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:39167 every 8 connection(s)
I20260806 22:33:19.810297 16302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:33:19.813720 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 16302
I20260806 22:33:19.814042 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:33:19.815135 16154 external_mini_cluster.cc:1428] Setting key 196bfa8792c3f31f9dcb542cfb345df9
Aug 06 22:33:19 dist-test-slave-h3ft krb5kdc[16162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055599, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
I20260806 22:33:19.842126 16491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:19.815289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:49535 (local address 127.15.198.190:40781)
0806 22:33:19.816426 (+ 1137us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:19.816434 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:19.817716 (+ 1282us) server_negotiation.cc:407] Connection header received
0806 22:33:19.820824 (+ 3108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:19.820830 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:19.820986 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:19.821142 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:19.823540 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:19.824915 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:19.827351 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:19.827839 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:19.828002 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:19.833608 (+ 5606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:19.833615 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:19.833619 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:19.833668 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:19.837378 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:19.838621 (+ 1243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:19.838626 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:19.838629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:19.838812 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:19.839445 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:19.839449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:19.839451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:19.839835 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:19.840022 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:19.841015 (+ 993us) server_negotiation.cc:299] Negotiation successful
0806 22:33:19.841322 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":899,"thread_start_us":721,"threads_started":1}
I20260806 22:33:19.843910 16494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:19.816117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:40781 (local address 127.15.198.129:49535)
0806 22:33:19.817516 (+ 1399us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:19.817569 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:19.820426 (+ 2857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:19.821508 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:19.821520 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:19.822315 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:19.823277 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:19.823308 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:19.825130 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:19.825138 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:19.827156 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:19.827172 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:19.828148 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:19.828158 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:19.828476 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:19.829456 (+ 980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:19.829492 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:19.833215 (+ 3723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:19.837637 (+ 4422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:19.837650 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:19.837672 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:19.838420 (+ 748us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:19.838959 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:19.838971 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:19.838974 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:19.839295 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:19.839992 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:19.840003 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:19.840509 (+ 506us) client_negotiation.cc:769] Sending connection context
0806 22:33:19.840903 (+ 394us) client_negotiation.cc:240] Negotiation successful
0806 22:33:19.841817 (+ 914us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1055,"thread_start_us":774,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
I20260806 22:33:19.846364 16489 heartbeater.cc:347] Connected to a master server at 127.15.198.190:40781
I20260806 22:33:19.846932 16489 heartbeater.cc:464] Registering TS with master...
I20260806 22:33:19.848448 16489 heartbeater.cc:511] Master 127.15.198.190:40781 requested a full tablet report, sending...
I20260806 22:33:19.853227 16229 ts_manager.cc:194] Registered new tserver with Master: 9dd91e5a816b4960895e3dce74d1cc94 (127.15.198.129:39167)
I20260806 22:33:19.856007 16229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:49535
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
I20260806 22:33:19.876091 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:40781
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:44421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:20.487432 16498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:20.487825 16498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:20.488018 16498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:20.497900 16498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:20.498113 16498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:20.498265 16498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:33:20.511927 16498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:44421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:40781
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:20.513924 16498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:20.515798 16498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:20.535636 16507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:20.535706 16514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:20.535728 16512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:20.537609 16498 server_base.cc:1034] running on GCE node
I20260806 22:33:20.538766 16498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:20.540457 16498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:20.541736 16498 hybrid_clock.cc:648] HybridClock initialized: now 1786055600541662 us; error 94 us; skew 500 ppm
I20260806 22:33:20.542152 16498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:20 dist-test-slave-h3ft krb5kdc[16162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055600, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:20.546615 16498 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:33:20.548799 16498 webserver.cc:460] Webserver started at http://127.15.198.130:43325/ using document root <none> and password file <none>
I20260806 22:33:20.549850 16498 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:20.549989 16498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:20.550395 16498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:20.554702 16498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "bd5b3df3a17448a0b8c423e46cdc2bf9"
format_stamp: "Formatted at 2026-08-06 22:33:20 on dist-test-slave-h3ft"
server_key: "4eb73cf0235b95c405f0035900a0d1e6"
server_key_iv: "e05c7154cdaea064f6088be5d4d0491b"
server_key_version: "encryptionkey@0"
I20260806 22:33:20.555735 16498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "bd5b3df3a17448a0b8c423e46cdc2bf9"
format_stamp: "Formatted at 2026-08-06 22:33:20 on dist-test-slave-h3ft"
server_key: "4eb73cf0235b95c405f0035900a0d1e6"
server_key_iv: "e05c7154cdaea064f6088be5d4d0491b"
server_key_version: "encryptionkey@0"
I20260806 22:33:20.564169 16498 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260806 22:33:20.570115 16526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:20.572431 16498 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 22:33:20.572829 16498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "bd5b3df3a17448a0b8c423e46cdc2bf9"
format_stamp: "Formatted at 2026-08-06 22:33:20 on dist-test-slave-h3ft"
server_key: "4eb73cf0235b95c405f0035900a0d1e6"
server_key_iv: "e05c7154cdaea064f6088be5d4d0491b"
server_key_version: "encryptionkey@0"
I20260806 22:33:20.573165 16498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:20.594126 16498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:20.601152 16498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:20.601647 16498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:20.603519 16498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:33:20.605876 16498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:20.605984 16498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:20.606107 16498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:20.606173 16498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:20.671871 16498 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:45423
I20260806 22:33:20.671911 16682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:45423 every 8 connection(s)
I20260806 22:33:20.673887 16498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:33:20.675809 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 16498
I20260806 22:33:20.676141 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:33:20.676844 16154 external_mini_cluster.cc:1428] Setting key 649d16da0971bfee2fda29732a8afbcc
Aug 06 22:33:20 dist-test-slave-h3ft krb5kdc[16162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055600, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
I20260806 22:33:20.709444 16686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:20.678473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:37099 (local address 127.15.198.190:40781)
0806 22:33:20.679260 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:20.679265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:20.681218 (+ 1953us) server_negotiation.cc:407] Connection header received
0806 22:33:20.684086 (+ 2868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:20.684097 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:20.684230 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:20.684376 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:20.686848 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:20.688487 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:20.691195 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:20.692422 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:20.692624 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:20.698448 (+ 5824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:20.698459 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:20.698466 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:20.698536 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:20.704358 (+ 5822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:20.705612 (+ 1254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:20.705634 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:20.705640 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:20.705909 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:20.706514 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:20.706524 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:20.706529 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:20.707096 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:20.707294 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:20.708461 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0806 22:33:20.708716 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":603,"threads_started":1}
I20260806 22:33:20.710866 16688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:20.679547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:40781 (local address 127.15.198.130:37099)
0806 22:33:20.680999 (+ 1452us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:20.681061 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:20.683673 (+ 2612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:20.684720 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:20.684733 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:20.685509 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:20.686578 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:20.686608 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:20.688798 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:20.688802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:20.690944 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:20.690963 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:20.692747 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:20.692753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:20.693078 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:20.694252 (+ 1174us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:20.694284 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:20.698042 (+ 3758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:20.704647 (+ 6605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:20.704659 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:20.704681 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:20.705394 (+ 713us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:20.706072 (+ 678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:20.706076 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:20.706078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:20.706361 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:20.707269 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:20.707277 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:20.707937 (+ 660us) client_negotiation.cc:769] Sending connection context
0806 22:33:20.708330 (+ 393us) client_negotiation.cc:240] Negotiation successful
0806 22:33:20.709047 (+ 717us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1035,"thread_start_us":660,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
I20260806 22:33:20.713478 16683 heartbeater.cc:347] Connected to a master server at 127.15.198.190:40781
I20260806 22:33:20.713932 16683 heartbeater.cc:464] Registering TS with master...
I20260806 22:33:20.715317 16683 heartbeater.cc:511] Master 127.15.198.190:40781 requested a full tablet report, sending...
I20260806 22:33:20.717918 16229 ts_manager.cc:194] Registered new tserver with Master: bd5b3df3a17448a0b8c423e46cdc2bf9 (127.15.198.130:45423)
I20260806 22:33:20.719638 16229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:37099
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
I20260806 22:33:20.748615 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:40781
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:44421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:33:20.861361 16489 heartbeater.cc:503] Master 127.15.198.190:40781 was elected leader, sending a full tablet report...
W20260806 22:33:21.341244 16692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:21.341754 16692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:21.341958 16692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:21.351344 16692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:21.351573 16692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:21.351785 16692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:33:21.365459 16692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:44421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:40781
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:21.367457 16692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:21.369474 16692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:21.388712 16699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:21.389009 16700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:21.389297 16702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:21.391214 16692 server_base.cc:1034] running on GCE node
I20260806 22:33:21.392098 16692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:21.393517 16692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:21.394860 16692 hybrid_clock.cc:648] HybridClock initialized: now 1786055601394726 us; error 70 us; skew 500 ppm
I20260806 22:33:21.395433 16692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:21 dist-test-slave-h3ft krb5kdc[16162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055601, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:21.400029 16692 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:33:21.402158 16692 webserver.cc:460] Webserver started at http://127.15.198.131:35859/ using document root <none> and password file <none>
I20260806 22:33:21.403383 16692 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:21.403571 16692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:21.403990 16692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:21.408288 16692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "3b126489a2e84a179c75b9189607e91c"
format_stamp: "Formatted at 2026-08-06 22:33:21 on dist-test-slave-h3ft"
server_key: "5d9a8fa2bbde3037389170ada965fc98"
server_key_iv: "fcea421861ecfb3f0087a0dc71235756"
server_key_version: "encryptionkey@0"
I20260806 22:33:21.409406 16692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "3b126489a2e84a179c75b9189607e91c"
format_stamp: "Formatted at 2026-08-06 22:33:21 on dist-test-slave-h3ft"
server_key: "5d9a8fa2bbde3037389170ada965fc98"
server_key_iv: "fcea421861ecfb3f0087a0dc71235756"
server_key_version: "encryptionkey@0"
I20260806 22:33:21.417474 16692 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 22:33:21.423274 16709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:21.425078 16692 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:33:21.425354 16692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "3b126489a2e84a179c75b9189607e91c"
format_stamp: "Formatted at 2026-08-06 22:33:21 on dist-test-slave-h3ft"
server_key: "5d9a8fa2bbde3037389170ada965fc98"
server_key_iv: "fcea421861ecfb3f0087a0dc71235756"
server_key_version: "encryptionkey@0"
I20260806 22:33:21.425683 16692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:21.442901 16692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:21.449108 16692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:21.449645 16692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:21.451264 16692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:33:21.453369 16692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:21.453554 16692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:21.453837 16692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:21.453987 16692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:21.520191 16692 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:36491
I20260806 22:33:21.520310 16866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:36491 every 8 connection(s)
I20260806 22:33:21.522018 16692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:33:21.526057 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 16692
I20260806 22:33:21.526358 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:33:21.527315 16154 external_mini_cluster.cc:1428] Setting key 77b0a58891f41a1d12bb5a87834fd6b2
Aug 06 22:33:21 dist-test-slave-h3ft krb5kdc[16162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055601, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:21.552435 16872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:21.526049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:39051 (local address 127.15.198.190:40781)
0806 22:33:21.527795 (+ 1746us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:21.527806 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:21.529369 (+ 1563us) server_negotiation.cc:407] Connection header received
0806 22:33:21.532444 (+ 3075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:21.532449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:21.532551 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:21.532664 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:21.534970 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:21.535978 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:21.538275 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:21.538739 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:21.538846 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:21.544061 (+ 5215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:21.544093 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:21.544101 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:21.544170 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:21.548519 (+ 4349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:21.549633 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:21.549637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:21.549640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:21.549778 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:21.550367 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:21.550370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:21.550372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:21.550707 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:21.550842 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:21.551830 (+ 988us) server_negotiation.cc:299] Negotiation successful
0806 22:33:21.552057 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1533,"thread_start_us":1360,"threads_started":1}
I20260806 22:33:21.554215 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:21.526747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:40781 (local address 127.15.198.131:39051)
0806 22:33:21.528620 (+ 1873us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:21.528696 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:21.532013 (+ 3317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:21.533053 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:21.533067 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:21.533823 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:21.534757 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:21.534782 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:21.536224 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:21.536230 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:21.538111 (+ 1881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:21.538127 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:21.539083 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:21.539092 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:21.539487 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:21.540350 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:21.540385 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:21.543627 (+ 3242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:21.548697 (+ 5070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:21.548708 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:21.548730 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:21.549510 (+ 780us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:21.549884 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:21.549888 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:21.549891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:21.550262 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:21.550826 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:21.550834 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:21.551418 (+ 584us) client_negotiation.cc:769] Sending connection context
0806 22:33:21.551742 (+ 324us) client_negotiation.cc:240] Negotiation successful
0806 22:33:21.552370 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1412,"thread_start_us":1079,"threads_started":1}
I20260806 22:33:21.556300 16869 heartbeater.cc:347] Connected to a master server at 127.15.198.190:40781
I20260806 22:33:21.556761 16869 heartbeater.cc:464] Registering TS with master...
I20260806 22:33:21.558286 16869 heartbeater.cc:511] Master 127.15.198.190:40781 requested a full tablet report, sending...
I20260806 22:33:21.560366 16229 ts_manager.cc:194] Registered new tserver with Master: 3b126489a2e84a179c75b9189607e91c (127.15.198.131:36491)
I20260806 22:33:21.561437 16229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:39051
I20260806 22:33:21.573388 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:33:21 dist-test-slave-h3ft krb5kdc[16162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055601, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:33:21 dist-test-slave-h3ft krb5kdc[16162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055601, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:33:21.649302 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:21.625029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41422 (local address 127.15.198.129:39167)
0806 22:33:21.626479 (+ 1450us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:21.626492 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:21.626684 (+ 192us) server_negotiation.cc:407] Connection header received
0806 22:33:21.627135 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:21.627146 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:21.627448 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:21.627716 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:21.630248 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:21.631940 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:21.633492 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:21.634420 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:21.634591 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:21.640617 (+ 6026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:21.640635 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:21.640657 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:21.640715 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:21.645065 (+ 4350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:21.646173 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:21.646183 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:21.646186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:21.646412 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:21.647073 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:21.647081 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:21.647084 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:21.647667 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:21.648000 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:21.648376 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 22:33:21.648674 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1144,"thread_start_us":733,"threads_started":1}
W20260806 22:33:21.651252 16362 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41422
Aug 06 22:33:21 dist-test-slave-h3ft krb5kdc[16162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:33:21 dist-test-slave-h3ft krb5kdc[16162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:33:21.720067 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:21.701604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41428 (local address 127.15.198.129:39167)
0806 22:33:21.701889 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:21.701896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:21.701924 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:33:21.702175 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:21.702181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:21.702273 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:21.702396 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:21.704165 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:21.705194 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:21.706651 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:21.707452 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:21.707610 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:21.712688 (+ 5078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:21.712694 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:21.712698 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:21.712746 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:21.716720 (+ 3974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:21.717568 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:21.717577 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:21.717585 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:21.717770 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:21.718207 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:21.718212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:21.718218 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:21.718555 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:21.718926 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:21.719345 (+ 419us) server_negotiation.cc:299] Negotiation successful
0806 22:33:21.719651 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260806 22:33:21.722275 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 16302
I20260806 22:33:21.724301 16683 heartbeater.cc:503] Master 127.15.198.190:40781 was elected leader, sending a full tablet report...
I20260806 22:33:21.744117 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:21.723901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41436 (local address 127.15.198.129:39167)
0806 22:33:21.724202 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:21.724212 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:21.724691 (+ 479us) server_negotiation.cc:407] Connection header received
0806 22:33:21.724918 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:21.724924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:21.725113 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:21.725305 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:21.726948 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:21.728069 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:21.729835 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:21.730472 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:21.730657 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:21.734781 (+ 4124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:21.734788 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:21.734792 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:21.734844 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:21.739902 (+ 5058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:21.741161 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:21.741167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:21.741170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:21.741395 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:21.741936 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:21.741941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:21.741943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:21.742403 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:21.742688 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:21.743195 (+ 507us) server_negotiation.cc:299] Negotiation successful
0806 22:33:21.743591 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260806 22:33:21.745504 16480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:21.844951 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 16302
I20260806 22:33:21.875202 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 16498
Aug 06 22:33:21 dist-test-slave-h3ft krb5kdc[16162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.130@KRBTEST.COM
I20260806 22:33:21.899348 16888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:21.876571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33550 (local address 127.15.198.130:45423)
0806 22:33:21.877349 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:21.877358 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:21.877392 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:33:21.877634 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:21.877643 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:21.877945 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:21.878208 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:21.880628 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:21.881898 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:21.884105 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:21.884908 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:21.885103 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:21.891240 (+ 6137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:21.891263 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:21.891287 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:21.891345 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:21.895439 (+ 4094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:21.896408 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:21.896420 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:21.896423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:21.896619 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:21.897160 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:21.897165 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:21.897168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:21.897653 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:21.898084 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:21.898446 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 22:33:21.898838 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":378,"threads_started":1}
I20260806 22:33:21.901260 16676 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:21.999953 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 16498
I20260806 22:33:22.032703 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 16692
Aug 06 22:33:22 dist-test-slave-h3ft krb5kdc[16162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.131@KRBTEST.COM
I20260806 22:33:22.052702 16890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:22.033900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49332 (local address 127.15.198.131:36491)
0806 22:33:22.034551 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:22.034561 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:22.034601 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:33:22.034681 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:22.034689 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:22.034975 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:22.035290 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:22.036844 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:22.038062 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:22.039552 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:22.040258 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:22.040419 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:22.045719 (+ 5300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:22.045737 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:22.045758 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:22.045808 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:22.049326 (+ 3518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:22.050246 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:22.050256 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:22.050258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:22.050407 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:22.050838 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:22.050842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:22.050844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:22.051363 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:22.051637 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:22.052010 (+ 373us) server_negotiation.cc:299] Negotiation successful
0806 22:33:22.052298 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":285,"threads_started":1}
I20260806 22:33:22.054538 16862 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:22.151129 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 16692
I20260806 22:33:22.183123 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 16185
I20260806 22:33:22.184793 16261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:22.307898 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 16185
2026-08-06T22:33:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (6325 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:33:22 dist-test-slave-h3ft krb5kdc[16896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:33:22 dist-test-slave-h3ft krb5kdc[16896](info): set up 2 sockets
Aug 06 22:33:22 dist-test-slave-h3ft krb5kdc[16896](info): commencing operation
krb5kdc: starting...
W20260806 22:33:23.668876 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.008s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:33:23 dist-test-slave-h3ft krb5kdc[16896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:33:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:33:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:33:23.943895 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:36869
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:44219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:36869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:24.379132 16921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:24.379588 16921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:24.379783 16921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:24.388252 16921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:33:24.388427 16921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:24.388478 16921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:24.388614 16921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:33:24.388716 16921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:33:24.400446 16921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:44219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:36869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:36869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:24.402441 16921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:24.404275 16921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:24.419270 16930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:24.419304 16929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:24.419348 16932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:24.421304 16921 server_base.cc:1034] running on GCE node
I20260806 22:33:24.422533 16921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:24.424750 16921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:24.426043 16921 hybrid_clock.cc:648] HybridClock initialized: now 1786055604425951 us; error 85 us; skew 500 ppm
I20260806 22:33:24.426568 16921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:24 dist-test-slave-h3ft krb5kdc[16896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055604, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:24.431250 16921 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:33:24.433605 16921 webserver.cc:460] Webserver started at http://127.15.198.190:44497/ using document root <none> and password file <none>
I20260806 22:33:24.434648 16921 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:24.434788 16921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:24.435261 16921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:24.440074 16921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "41681ed80d544831ac9b2d678d2281ce"
format_stamp: "Formatted at 2026-08-06 22:33:24 on dist-test-slave-h3ft"
server_key: "bbd896dfc53fffa8f396311d1dc8a987"
server_key_iv: "76b1f3ee36e76cbda8e6330a6ce56ba1"
server_key_version: "encryptionkey@0"
I20260806 22:33:24.441196 16921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "41681ed80d544831ac9b2d678d2281ce"
format_stamp: "Formatted at 2026-08-06 22:33:24 on dist-test-slave-h3ft"
server_key: "bbd896dfc53fffa8f396311d1dc8a987"
server_key_iv: "76b1f3ee36e76cbda8e6330a6ce56ba1"
server_key_version: "encryptionkey@0"
I20260806 22:33:24.448962 16921 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260806 22:33:24.453926 16944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:24.456110 16921 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:33:24.456403 16921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "41681ed80d544831ac9b2d678d2281ce"
format_stamp: "Formatted at 2026-08-06 22:33:24 on dist-test-slave-h3ft"
server_key: "bbd896dfc53fffa8f396311d1dc8a987"
server_key_iv: "76b1f3ee36e76cbda8e6330a6ce56ba1"
server_key_version: "encryptionkey@0"
I20260806 22:33:24.456705 16921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:24.475131 16921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:24.481352 16921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:24.481786 16921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:24.506517 16921 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:36869
I20260806 22:33:24.506544 17012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:36869 every 8 connection(s)
I20260806 22:33:24.508514 16921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:33:24.514245 17014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:24.515672 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 16921
I20260806 22:33:24.516000 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:33:24.516824 16154 external_mini_cluster.cc:1428] Setting key 91f2bcf5ef15d582d9bc1b3737e283ad
I20260806 22:33:24.534802 17014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce: Bootstrap starting.
Aug 06 22:33:24 dist-test-slave-h3ft krb5kdc[16896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:24.540272 17014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:24.542346 17014 log.cc:824] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce: Log is configured to *not* fsync() on all Append() calls
I20260806 22:33:24.547431 17014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce: No bootstrap required, opened a new log
I20260806 22:33:24.551700 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:24.520096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33280 (local address 127.15.198.190:36869)
0806 22:33:24.521399 (+ 1303us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:24.521422 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:24.521505 (+ 83us) server_negotiation.cc:407] Connection header received
0806 22:33:24.523947 (+ 2442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:24.524000 (+ 53us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:24.524884 (+ 884us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:24.525556 (+ 672us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:24.527621 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:24.530152 (+ 2531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:24.532162 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:24.533609 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:24.533916 (+ 307us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:24.540210 (+ 6294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:24.540231 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:24.540271 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:24.540350 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:24.545469 (+ 5119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:24.546720 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:24.546732 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:24.546744 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:24.547100 (+ 356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:24.547645 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:24.547651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:24.547659 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:24.548424 (+ 765us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:24.548844 (+ 420us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:24.549567 (+ 723us) server_negotiation.cc:299] Negotiation successful
0806 22:33:24.550170 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":948,"thread_start_us":569,"threads_started":1}
I20260806 22:33:24.558449 17014 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41681ed80d544831ac9b2d678d2281ce" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36869 } }
I20260806 22:33:24.559088 17014 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:24.559242 17014 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41681ed80d544831ac9b2d678d2281ce, State: Initialized, Role: FOLLOWER
I20260806 22:33:24.560082 17014 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41681ed80d544831ac9b2d678d2281ce" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36869 } }
I20260806 22:33:24.560422 17014 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:33:24.560693 17014 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:33:24.560956 17014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:24.564222 17014 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41681ed80d544831ac9b2d678d2281ce" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36869 } }
I20260806 22:33:24.565047 17014 leader_election.cc:302] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41681ed80d544831ac9b2d678d2281ce; no voters:
I20260806 22:33:24.565867 17014 leader_election.cc:288] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:33:24.566164 17023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:33:24.567695 17023 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [term 1 LEADER]: Becoming Leader. State: Replica: 41681ed80d544831ac9b2d678d2281ce, State: Running, Role: LEADER
I20260806 22:33:24.568508 17023 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41681ed80d544831ac9b2d678d2281ce" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36869 } }
I20260806 22:33:24.569136 17014 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:33:24.572561 17024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41681ed80d544831ac9b2d678d2281ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41681ed80d544831ac9b2d678d2281ce" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36869 } } }
I20260806 22:33:24.573277 17024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [sys.catalog]: This master's current role is: LEADER
I20260806 22:33:24.573442 17025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41681ed80d544831ac9b2d678d2281ce. Latest consensus state: current_term: 1 leader_uuid: "41681ed80d544831ac9b2d678d2281ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41681ed80d544831ac9b2d678d2281ce" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36869 } } }
I20260806 22:33:24.573932 17025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce [sys.catalog]: This master's current role is: LEADER
I20260806 22:33:24.576745 17031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:33:24.584728 17031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:33:24.597666 17031 catalog_manager.cc:1347] Generated new cluster ID: 2c5a730db9654148afca84b57eb5a45c
I20260806 22:33:24.597856 17031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:33:24.608175 17031 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:33:24.610675 17031 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:33:24.633569 17031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41681ed80d544831ac9b2d678d2281ce: Generated new TSK 0
I20260806 22:33:24.635046 17031 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
I20260806 22:33:24.704272 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36869
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:44219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:25.178201 17050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:25.178644 17050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:25.178835 17050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:25.187127 17050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:25.187290 17050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:25.187520 17050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:33:25.199121 17050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:44219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36869
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:25.201045 17050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:25.202858 17050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:25.219099 17055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:25.219377 17056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:25.219097 17058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:25.220592 17050 server_base.cc:1034] running on GCE node
I20260806 22:33:25.221457 17050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:25.222843 17050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:25.224134 17050 hybrid_clock.cc:648] HybridClock initialized: now 1786055605224083 us; error 51 us; skew 500 ppm
I20260806 22:33:25.224581 17050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:25 dist-test-slave-h3ft krb5kdc[16896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055605, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:25.228966 17050 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:33:25.231146 17050 webserver.cc:460] Webserver started at http://127.15.198.129:34533/ using document root <none> and password file <none>
I20260806 22:33:25.232151 17050 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:25.232334 17050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:25.232859 17050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:25.236959 17050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "abca76eee59f44919f14e34b4c5fb73c"
format_stamp: "Formatted at 2026-08-06 22:33:25 on dist-test-slave-h3ft"
server_key: "4d2476b8d7259e0d6c65b01817832c93"
server_key_iv: "21819c02ed2a29901ac0d845898d256b"
server_key_version: "encryptionkey@0"
I20260806 22:33:25.238088 17050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "abca76eee59f44919f14e34b4c5fb73c"
format_stamp: "Formatted at 2026-08-06 22:33:25 on dist-test-slave-h3ft"
server_key: "4d2476b8d7259e0d6c65b01817832c93"
server_key_iv: "21819c02ed2a29901ac0d845898d256b"
server_key_version: "encryptionkey@0"
I20260806 22:33:25.245319 17050 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 22:33:25.250310 17065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:25.252321 17050 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 22:33:25.252525 17050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "abca76eee59f44919f14e34b4c5fb73c"
format_stamp: "Formatted at 2026-08-06 22:33:25 on dist-test-slave-h3ft"
server_key: "4d2476b8d7259e0d6c65b01817832c93"
server_key_iv: "21819c02ed2a29901ac0d845898d256b"
server_key_version: "encryptionkey@0"
I20260806 22:33:25.252774 17050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:25.272969 17050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:25.279066 17050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:25.279398 17050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:25.280848 17050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:33:25.282627 17050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:25.282716 17050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:25.282927 17050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:25.283100 17050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260806 22:33:25.325253 17050 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:36213
I20260806 22:33:25.325302 17213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:36213 every 8 connection(s)
I20260806 22:33:25.327101 17050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:33:25.331683 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 17050
I20260806 22:33:25.332034 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:33:25.333006 16154 external_mini_cluster.cc:1428] Setting key 670e5c92fd0fb427464f9a323da906b9
Aug 06 22:33:25 dist-test-slave-h3ft krb5kdc[16896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055605, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:25.356144 17221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:25.330734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:40629 (local address 127.15.198.190:36869)
0806 22:33:25.331453 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:25.331459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:25.332839 (+ 1380us) server_negotiation.cc:407] Connection header received
0806 22:33:25.335799 (+ 2960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:25.335804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:25.335933 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:25.336075 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:25.338097 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:25.339394 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:25.341347 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:25.342082 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:25.342245 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:25.347815 (+ 5570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:25.347822 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:25.347826 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:25.347875 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:25.351251 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:25.352579 (+ 1328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:25.352584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:25.352587 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:25.352749 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:25.353393 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:25.353397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:25.353400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:25.353771 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:25.353912 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:25.355522 (+ 1610us) server_negotiation.cc:299] Negotiation successful
0806 22:33:25.355788 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":442,"threads_started":1}
I20260806 22:33:25.358232 17222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:25.331500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:36869 (local address 127.15.198.129:40629)
0806 22:33:25.332599 (+ 1099us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:25.332660 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:25.335337 (+ 2677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:25.336378 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:25.336389 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:25.337105 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:25.337873 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:25.337896 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:25.339611 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:25.339616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:25.341146 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:25.341163 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:25.342391 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:25.342397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:25.342672 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:25.343669 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:25.343703 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:25.347442 (+ 3739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:25.351498 (+ 4056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:25.351516 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:25.351550 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:25.352399 (+ 849us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:25.352876 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:25.352884 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:25.352891 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:25.353231 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:25.353935 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:25.353949 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:25.354659 (+ 710us) client_negotiation.cc:769] Sending connection context
0806 22:33:25.355117 (+ 458us) client_negotiation.cc:240] Negotiation successful
0806 22:33:25.355992 (+ 875us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"thread_start_us":420,"threads_started":1}
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
I20260806 22:33:25.361016 17214 heartbeater.cc:347] Connected to a master server at 127.15.198.190:36869
I20260806 22:33:25.361482 17214 heartbeater.cc:464] Registering TS with master...
I20260806 22:33:25.362587 17214 heartbeater.cc:511] Master 127.15.198.190:36869 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
I20260806 22:33:25.365957 16964 ts_manager.cc:194] Registered new tserver with Master: abca76eee59f44919f14e34b4c5fb73c (127.15.198.129:36213)
I20260806 22:33:25.367966 16964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:40629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
I20260806 22:33:25.395412 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36869
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:44219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:25.824427 17225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:25.824873 17225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:25.824999 17225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:25.833827 17225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:25.834012 17225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:25.834151 17225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:33:25.846143 17225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:44219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36869
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:25.848260 17225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:25.850006 17225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:25.867668 17231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:25.867623 17232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:25.867697 17234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:25.868856 17225 server_base.cc:1034] running on GCE node
I20260806 22:33:25.869724 17225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:25.871136 17225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:25.872382 17225 hybrid_clock.cc:648] HybridClock initialized: now 1786055605872342 us; error 46 us; skew 500 ppm
I20260806 22:33:25.872749 17225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:25 dist-test-slave-h3ft krb5kdc[16896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055605, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:25.877395 17225 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:33:25.880389 17225 webserver.cc:460] Webserver started at http://127.15.198.130:42899/ using document root <none> and password file <none>
I20260806 22:33:25.881883 17225 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:25.882092 17225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:25.882644 17225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:25.889082 17225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "bd2324b6b9e740919703b90b8d4fd476"
format_stamp: "Formatted at 2026-08-06 22:33:25 on dist-test-slave-h3ft"
server_key: "84a79c43ab0345466436aaccc016c4ad"
server_key_iv: "bf9684d5d88cec1413f7b5a10b46829b"
server_key_version: "encryptionkey@0"
I20260806 22:33:25.890689 17225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "bd2324b6b9e740919703b90b8d4fd476"
format_stamp: "Formatted at 2026-08-06 22:33:25 on dist-test-slave-h3ft"
server_key: "84a79c43ab0345466436aaccc016c4ad"
server_key_iv: "bf9684d5d88cec1413f7b5a10b46829b"
server_key_version: "encryptionkey@0"
I20260806 22:33:25.898646 17225 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260806 22:33:25.903644 17245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:25.905652 17225 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 22:33:25.905867 17225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "bd2324b6b9e740919703b90b8d4fd476"
format_stamp: "Formatted at 2026-08-06 22:33:25 on dist-test-slave-h3ft"
server_key: "84a79c43ab0345466436aaccc016c4ad"
server_key_iv: "bf9684d5d88cec1413f7b5a10b46829b"
server_key_version: "encryptionkey@0"
I20260806 22:33:25.906168 17225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:25.954088 17225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:25.960918 17225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:25.961400 17225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:25.962870 17225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:33:25.964810 17225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:25.964939 17225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:25.965131 17225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:25.965219 17225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:26.008682 17225 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:45767
I20260806 22:33:26.008728 17401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:45767 every 8 connection(s)
I20260806 22:33:26.010576 17225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:33:26.017805 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 17225
I20260806 22:33:26.018121 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:33:26.018846 16154 external_mini_cluster.cc:1428] Setting key ae8db66981296f6c4e1c80e6ea3cee87
Aug 06 22:33:26 dist-test-slave-h3ft krb5kdc[16896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055605, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:26.040411 17404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:26.015368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:58983 (local address 127.15.198.190:36869)
0806 22:33:26.016072 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:26.016080 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:26.021465 (+ 5385us) server_negotiation.cc:407] Connection header received
0806 22:33:26.021540 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:26.021545 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:26.021631 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:26.021787 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:26.024152 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.025175 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.027184 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.027679 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.027841 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:26.032630 (+ 4789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:26.032637 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:26.032641 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:26.032685 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:26.036256 (+ 3571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:26.037314 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:26.037319 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:26.037322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:26.037487 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:26.038044 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:26.038049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:26.038051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:26.038417 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:26.038617 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:26.039637 (+ 1020us) server_negotiation.cc:299] Negotiation successful
0806 22:33:26.039956 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":324,"threads_started":1}
I20260806 22:33:26.041857 17405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:26.016301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:36869 (local address 127.15.198.130:58983)
0806 22:33:26.017904 (+ 1603us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:26.018031 (+ 127us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:26.020991 (+ 2960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:26.022171 (+ 1180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:26.022183 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:26.022963 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:26.023888 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:26.023912 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.025392 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.025397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:26.026930 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:26.026952 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.027969 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.027978 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:26.028274 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:26.029193 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:26.029221 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:26.032298 (+ 3077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:26.036498 (+ 4200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:26.036509 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:26.036531 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:26.037135 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:26.037623 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:26.037629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:26.037633 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:26.037894 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:26.038585 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:26.038596 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:26.039170 (+ 574us) client_negotiation.cc:769] Sending connection context
0806 22:33:26.039511 (+ 341us) client_negotiation.cc:240] Negotiation successful
0806 22:33:26.040251 (+ 740us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1170,"thread_start_us":497,"threads_started":1}
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
I20260806 22:33:26.044049 17402 heartbeater.cc:347] Connected to a master server at 127.15.198.190:36869
I20260806 22:33:26.044477 17402 heartbeater.cc:464] Registering TS with master...
I20260806 22:33:26.045511 17402 heartbeater.cc:511] Master 127.15.198.190:36869 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
I20260806 22:33:26.047750 16964 ts_manager.cc:194] Registered new tserver with Master: bd2324b6b9e740919703b90b8d4fd476 (127.15.198.130:45767)
I20260806 22:33:26.048836 16964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:58983
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
I20260806 22:33:26.096084 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36869
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:44219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:33:26.372865 17214 heartbeater.cc:503] Master 127.15.198.190:36869 was elected leader, sending a full tablet report...
W20260806 22:33:26.527999 17409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:26.528424 17409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:26.528599 17409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:26.537236 17409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:26.537413 17409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:26.537626 17409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:33:26.549552 17409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:44219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36869
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:26.551453 17409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:26.553198 17409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:26.570716 17415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:26.570735 17414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:26.570964 17417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:26.571964 17409 server_base.cc:1034] running on GCE node
I20260806 22:33:26.572906 17409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:26.574353 17409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:26.575670 17409 hybrid_clock.cc:648] HybridClock initialized: now 1786055606575579 us; error 83 us; skew 500 ppm
I20260806 22:33:26.576354 17409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:26 dist-test-slave-h3ft krb5kdc[16896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055606, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:26.581650 17409 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:33:26.584419 17409 webserver.cc:460] Webserver started at http://127.15.198.131:36145/ using document root <none> and password file <none>
I20260806 22:33:26.585688 17409 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:26.585915 17409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:26.586411 17409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:26.591229 17409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "c03eb14b353e4dbea18b72ec10bf0481"
format_stamp: "Formatted at 2026-08-06 22:33:26 on dist-test-slave-h3ft"
server_key: "b4e3769bb453d388e4dae48ec6dfbd8d"
server_key_iv: "75f3007593b7db6ced48f1d107db6a74"
server_key_version: "encryptionkey@0"
I20260806 22:33:26.592362 17409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "c03eb14b353e4dbea18b72ec10bf0481"
format_stamp: "Formatted at 2026-08-06 22:33:26 on dist-test-slave-h3ft"
server_key: "b4e3769bb453d388e4dae48ec6dfbd8d"
server_key_iv: "75f3007593b7db6ced48f1d107db6a74"
server_key_version: "encryptionkey@0"
I20260806 22:33:26.600307 17409 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 22:33:26.606645 17427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:26.609089 17409 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260806 22:33:26.609411 17409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "c03eb14b353e4dbea18b72ec10bf0481"
format_stamp: "Formatted at 2026-08-06 22:33:26 on dist-test-slave-h3ft"
server_key: "b4e3769bb453d388e4dae48ec6dfbd8d"
server_key_iv: "75f3007593b7db6ced48f1d107db6a74"
server_key_version: "encryptionkey@0"
I20260806 22:33:26.609789 17409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:26.642814 17409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:26.650321 17409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:26.650815 17409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:26.652604 17409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:33:26.654670 17409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:26.654809 17409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:26.655014 17409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:26.655099 17409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 22:33:26.721383 17409 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:43729
I20260806 22:33:26.721516 17579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:43729 every 8 connection(s)
I20260806 22:33:26.723455 17409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:33:26.729506 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 17409
I20260806 22:33:26.729976 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:33:26.730894 16154 external_mini_cluster.cc:1428] Setting key 9ec95cb19e79f9a2cef0cea4ecf597a7
Aug 06 22:33:26 dist-test-slave-h3ft krb5kdc[16896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055606, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:26.756645 17582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:26.727988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:35343 (local address 127.15.198.190:36869)
0806 22:33:26.728635 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:26.728640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:26.730866 (+ 2226us) server_negotiation.cc:407] Connection header received
0806 22:33:26.734623 (+ 3757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:26.734630 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:26.734748 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:26.734915 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:26.737713 (+ 2798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.738914 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.741002 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.741494 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.741604 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:26.747450 (+ 5846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:26.747458 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:26.747465 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:26.747528 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:26.752022 (+ 4494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:26.753187 (+ 1165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:26.753193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:26.753196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:26.753497 (+ 301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:26.754060 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:26.754065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:26.754068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:26.754566 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:26.754708 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:26.755890 (+ 1182us) server_negotiation.cc:299] Negotiation successful
0806 22:33:26.756150 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":309,"threads_started":1}
I20260806 22:33:26.757892 17583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:26.728780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:36869 (local address 127.15.198.131:35343)
0806 22:33:26.730566 (+ 1786us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:26.730635 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:26.734260 (+ 3625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:26.735382 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:26.735399 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:26.736519 (+ 1120us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:26.737469 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:26.737502 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.739205 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.739210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:26.740817 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:26.740848 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.741858 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.741866 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:26.742261 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:26.743449 (+ 1188us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:26.743480 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:26.747116 (+ 3636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:26.752240 (+ 5124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:26.752252 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:26.752275 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:26.752989 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:26.753637 (+ 648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:26.753641 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:26.753643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:26.753904 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:26.754696 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:26.754704 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:26.755294 (+ 590us) client_negotiation.cc:769] Sending connection context
0806 22:33:26.755647 (+ 353us) client_negotiation.cc:240] Negotiation successful
0806 22:33:26.756230 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1263,"thread_start_us":758,"threads_started":1}
I20260806 22:33:26.760282 17580 heartbeater.cc:347] Connected to a master server at 127.15.198.190:36869
I20260806 22:33:26.760802 17580 heartbeater.cc:464] Registering TS with master...
I20260806 22:33:26.762322 17580 heartbeater.cc:511] Master 127.15.198.190:36869 requested a full tablet report, sending...
I20260806 22:33:26.764801 16964 ts_manager.cc:194] Registered new tserver with Master: c03eb14b353e4dbea18b72ec10bf0481 (127.15.198.131:43729)
I20260806 22:33:26.766029 16964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:35343
I20260806 22:33:26.776641 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:33:26.805478 17582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:26.786805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33286 (local address 127.15.198.190:36869)
0806 22:33:26.787185 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:26.787192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:26.787920 (+ 728us) server_negotiation.cc:407] Connection header received
0806 22:33:26.788235 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:26.788241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:26.788360 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:26.788493 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:26.790818 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.791953 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.793796 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.794287 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.794468 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:26.797985 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:26.797991 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:26.797995 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:26.798047 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:26.801576 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:26.802516 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:26.802522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:26.802533 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:26.802694 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:26.803245 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:26.803257 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:26.803260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:26.803861 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:26.804226 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:26.804677 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 22:33:26.805070 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260806 22:33:26.816386 16964 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:33:26.826304 16964 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:33:26.872030 17602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:26.846918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:36213 (local address 127.0.0.1:35570)
0806 22:33:26.853484 (+ 6566us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:26.853617 (+ 133us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:26.853944 (+ 327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:26.855164 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:26.855178 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:26.855234 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:26.856202 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:26.856231 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.860852 (+ 4621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.860863 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:26.865408 (+ 4545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:26.865482 (+ 74us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.869921 (+ 4439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.869934 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:26.870396 (+ 462us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:26.870505 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 22:33:26.870796 (+ 291us) client_negotiation.cc:240] Negotiation successful
0806 22:33:26.871237 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6138,"thread_start_us":5902,"threads_started":1}
I20260806 22:33:26.872475 17604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:26.848326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35570 (local address 127.15.198.129:36213)
0806 22:33:26.852618 (+ 4292us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:26.852630 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:26.854222 (+ 1592us) server_negotiation.cc:407] Connection header received
0806 22:33:26.854389 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:26.854400 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:26.854775 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:26.856631 (+ 1856us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:26.857234 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.860552 (+ 3318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.865925 (+ 5373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.869219 (+ 3294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.871358 (+ 2139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:26.871649 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:26.871761 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 22:33:26.872033 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4029,"thread_start_us":3793,"threads_started":1}
I20260806 22:33:26.879506 17133 tablet_service.cc:1467] Processing CreateTablet for tablet 4187f0b7e7d64839b926e2cc9e45741d (DEFAULT_TABLE table=test-table [id=bab10dcf407641359b69ecad3afc9db6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:26.882258 17133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4187f0b7e7d64839b926e2cc9e45741d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:26.883291 17607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:26.857396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42118 (local address 127.15.198.131:43729)
0806 22:33:26.871164 (+ 13768us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:26.871177 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:26.872656 (+ 1479us) server_negotiation.cc:407] Connection header received
0806 22:33:26.872799 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:26.872810 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:26.873146 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:26.873423 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:26.874173 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.877021 (+ 2848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.880010 (+ 2989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.881532 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.881761 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:26.881940 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:26.882650 (+ 710us) server_negotiation.cc:299] Negotiation successful
0806 22:33:26.882861 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12986,"thread_start_us":13309,"threads_started":1}
I20260806 22:33:26.883407 17602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:26.854611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:43729 (local address 127.0.0.1:42118)
0806 22:33:26.872357 (+ 17746us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:26.872377 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:26.872489 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:26.873442 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:26.873449 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:26.873492 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:26.873998 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:26.874024 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.877234 (+ 3210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.877239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:26.879851 (+ 2612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:26.879862 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.882044 (+ 2182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.882049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:26.882263 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:26.882291 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 22:33:26.882424 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 22:33:26.882663 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17645,"mutex_wait_us":23,"thread_start_us":18447,"threads_started":1}
I20260806 22:33:26.884747 17603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:26.848362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49970 (local address 127.15.198.130:45767)
0806 22:33:26.851833 (+ 3471us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:26.851849 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:26.871310 (+ 19461us) server_negotiation.cc:407] Connection header received
0806 22:33:26.871481 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:26.871492 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:26.872008 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:26.872283 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:26.874118 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.875806 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.879905 (+ 4099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.881820 (+ 1915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.882096 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:26.882283 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:26.883975 (+ 1692us) server_negotiation.cc:299] Negotiation successful
0806 22:33:26.884270 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3221,"thread_start_us":3035,"threads_started":1}
I20260806 22:33:26.885098 17605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:26.849048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:45767 (local address 127.0.0.1:49970)
0806 22:33:26.870860 (+ 21812us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:26.870892 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:26.871095 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:26.872674 (+ 1579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:26.872682 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:26.872716 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:26.873862 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:26.873882 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.876062 (+ 2180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.876067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:26.879604 (+ 3537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:26.879621 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:26.882461 (+ 2840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:26.882466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:26.882665 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:26.882708 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:33:26.882862 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 22:33:26.883051 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19102,"mutex_wait_us":2,"thread_start_us":21429,"threads_started":1}
I20260806 22:33:26.889899 17491 tablet_service.cc:1467] Processing CreateTablet for tablet 4187f0b7e7d64839b926e2cc9e45741d (DEFAULT_TABLE table=test-table [id=bab10dcf407641359b69ecad3afc9db6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:26.889875 17310 tablet_service.cc:1467] Processing CreateTablet for tablet 4187f0b7e7d64839b926e2cc9e45741d (DEFAULT_TABLE table=test-table [id=bab10dcf407641359b69ecad3afc9db6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:26.892091 17491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4187f0b7e7d64839b926e2cc9e45741d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:26.892154 17310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4187f0b7e7d64839b926e2cc9e45741d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:26.900092 17608 tablet_bootstrap.cc:492] T 4187f0b7e7d64839b926e2cc9e45741d P abca76eee59f44919f14e34b4c5fb73c: Bootstrap starting.
I20260806 22:33:26.907254 17608 tablet_bootstrap.cc:654] T 4187f0b7e7d64839b926e2cc9e45741d P abca76eee59f44919f14e34b4c5fb73c: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:26.909142 17608 log.cc:824] T 4187f0b7e7d64839b926e2cc9e45741d P abca76eee59f44919f14e34b4c5fb73c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:33:26.913894 17608 tablet_bootstrap.cc:492] T 4187f0b7e7d64839b926e2cc9e45741d P abca76eee59f44919f14e34b4c5fb73c: No bootstrap required, opened a new log
I20260806 22:33:26.913950 17610 tablet_bootstrap.cc:492] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476: Bootstrap starting.
I20260806 22:33:26.914625 17608 ts_tablet_manager.cc:1397] T 4187f0b7e7d64839b926e2cc9e45741d P abca76eee59f44919f14e34b4c5fb73c: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.007s
I20260806 22:33:26.915916 17611 tablet_bootstrap.cc:492] T 4187f0b7e7d64839b926e2cc9e45741d P c03eb14b353e4dbea18b72ec10bf0481: Bootstrap starting.
I20260806 22:33:26.921427 17611 tablet_bootstrap.cc:654] T 4187f0b7e7d64839b926e2cc9e45741d P c03eb14b353e4dbea18b72ec10bf0481: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:26.921638 17610 tablet_bootstrap.cc:654] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:26.923350 17611 log.cc:824] T 4187f0b7e7d64839b926e2cc9e45741d P c03eb14b353e4dbea18b72ec10bf0481: Log is configured to *not* fsync() on all Append() calls
I20260806 22:33:26.924003 17610 log.cc:824] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476: Log is configured to *not* fsync() on all Append() calls
I20260806 22:33:26.928911 17611 tablet_bootstrap.cc:492] T 4187f0b7e7d64839b926e2cc9e45741d P c03eb14b353e4dbea18b72ec10bf0481: No bootstrap required, opened a new log
I20260806 22:33:26.929431 17611 ts_tablet_manager.cc:1397] T 4187f0b7e7d64839b926e2cc9e45741d P c03eb14b353e4dbea18b72ec10bf0481: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.003s
I20260806 22:33:26.930331 17608 raft_consensus.cc:348] T 4187f0b7e7d64839b926e2cc9e45741d P abca76eee59f44919f14e34b4c5fb73c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abca76eee59f44919f14e34b4c5fb73c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 36213 } } peers { permanent_uuid: "c03eb14b353e4dbea18b72ec10bf0481" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43729 } } peers { permanent_uuid: "bd2324b6b9e740919703b90b8d4fd476" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45767 } }
I20260806 22:33:26.931032 17608 raft_consensus.cc:374] T 4187f0b7e7d64839b926e2cc9e45741d P abca76eee59f44919f14e34b4c5fb73c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:26.931239 17608 raft_consensus.cc:729] T 4187f0b7e7d64839b926e2cc9e45741d P abca76eee59f44919f14e34b4c5fb73c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abca76eee59f44919f14e34b4c5fb73c, State: Initialized, Role: FOLLOWER
I20260806 22:33:26.932144 17610 tablet_bootstrap.cc:492] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476: No bootstrap required, opened a new log
I20260806 22:33:26.932730 17610 ts_tablet_manager.cc:1397] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.005s
I20260806 22:33:26.932446 17608 consensus_queue.cc:260] T 4187f0b7e7d64839b926e2cc9e45741d P abca76eee59f44919f14e34b4c5fb73c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abca76eee59f44919f14e34b4c5fb73c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 36213 } } peers { permanent_uuid: "c03eb14b353e4dbea18b72ec10bf0481" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43729 } } peers { permanent_uuid: "bd2324b6b9e740919703b90b8d4fd476" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45767 } }
I20260806 22:33:26.935598 17608 ts_tablet_manager.cc:1428] T 4187f0b7e7d64839b926e2cc9e45741d P abca76eee59f44919f14e34b4c5fb73c: Time spent starting tablet: real 0.021s user 0.019s sys 0.000s
I20260806 22:33:26.943759 17611 raft_consensus.cc:348] T 4187f0b7e7d64839b926e2cc9e45741d P c03eb14b353e4dbea18b72ec10bf0481 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abca76eee59f44919f14e34b4c5fb73c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 36213 } } peers { permanent_uuid: "c03eb14b353e4dbea18b72ec10bf0481" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43729 } } peers { permanent_uuid: "bd2324b6b9e740919703b90b8d4fd476" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45767 } }
I20260806 22:33:26.944412 17611 raft_consensus.cc:374] T 4187f0b7e7d64839b926e2cc9e45741d P c03eb14b353e4dbea18b72ec10bf0481 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:26.944621 17611 raft_consensus.cc:729] T 4187f0b7e7d64839b926e2cc9e45741d P c03eb14b353e4dbea18b72ec10bf0481 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c03eb14b353e4dbea18b72ec10bf0481, State: Initialized, Role: FOLLOWER
I20260806 22:33:26.945672 17611 consensus_queue.cc:260] T 4187f0b7e7d64839b926e2cc9e45741d P c03eb14b353e4dbea18b72ec10bf0481 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abca76eee59f44919f14e34b4c5fb73c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 36213 } } peers { permanent_uuid: "c03eb14b353e4dbea18b72ec10bf0481" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43729 } } peers { permanent_uuid: "bd2324b6b9e740919703b90b8d4fd476" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45767 } }
I20260806 22:33:26.948342 17580 heartbeater.cc:503] Master 127.15.198.190:36869 was elected leader, sending a full tablet report...
I20260806 22:33:26.948887 17611 ts_tablet_manager.cc:1428] T 4187f0b7e7d64839b926e2cc9e45741d P c03eb14b353e4dbea18b72ec10bf0481: Time spent starting tablet: real 0.019s user 0.010s sys 0.008s
I20260806 22:33:26.949610 17610 raft_consensus.cc:348] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abca76eee59f44919f14e34b4c5fb73c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 36213 } } peers { permanent_uuid: "c03eb14b353e4dbea18b72ec10bf0481" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43729 } } peers { permanent_uuid: "bd2324b6b9e740919703b90b8d4fd476" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45767 } }
I20260806 22:33:26.950183 17610 raft_consensus.cc:374] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:26.950353 17610 raft_consensus.cc:729] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd2324b6b9e740919703b90b8d4fd476, State: Initialized, Role: FOLLOWER
I20260806 22:33:26.951400 17610 consensus_queue.cc:260] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abca76eee59f44919f14e34b4c5fb73c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 36213 } } peers { permanent_uuid: "c03eb14b353e4dbea18b72ec10bf0481" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43729 } } peers { permanent_uuid: "bd2324b6b9e740919703b90b8d4fd476" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45767 } }
I20260806 22:33:26.953881 17610 ts_tablet_manager.cc:1428] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476: Time spent starting tablet: real 0.021s user 0.008s sys 0.010s
I20260806 22:33:26.954257 17402 heartbeater.cc:503] Master 127.15.198.190:36869 was elected leader, sending a full tablet report...
W20260806 22:33:26.977005 17581 tablet.cc:2369] T 4187f0b7e7d64839b926e2cc9e45741d P c03eb14b353e4dbea18b72ec10bf0481: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:33:27.002902 17616 raft_consensus.cc:482] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:33:27.003537 17616 raft_consensus.cc:504] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abca76eee59f44919f14e34b4c5fb73c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 36213 } } peers { permanent_uuid: "c03eb14b353e4dbea18b72ec10bf0481" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43729 } } peers { permanent_uuid: "bd2324b6b9e740919703b90b8d4fd476" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45767 } }
I20260806 22:33:27.006537 17616 leader_election.cc:288] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abca76eee59f44919f14e34b4c5fb73c (127.15.198.129:36213), c03eb14b353e4dbea18b72ec10bf0481 (127.15.198.131:43729)
W20260806 22:33:27.014714 17403 tablet.cc:2369] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:33:27.019035 17604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:27.007281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:51825 (local address 127.15.198.129:36213)
0806 22:33:27.007730 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:27.007741 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:27.008779 (+ 1038us) server_negotiation.cc:407] Connection header received
0806 22:33:27.009215 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:27.009223 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:27.009340 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:27.009439 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:27.010857 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.012484 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.016538 (+ 4054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.017719 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.017948 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:27.018056 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:27.018461 (+ 405us) server_negotiation.cc:299] Negotiation successful
0806 22:33:27.018624 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 22:33:27.019088 17619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:27.007164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:36213 (local address 127.15.198.130:51825)
0806 22:33:27.008592 (+ 1428us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:27.008620 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:27.008978 (+ 358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:27.009736 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:27.009741 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:27.009784 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:27.010625 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:27.010644 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.012666 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.012674 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:27.016243 (+ 3569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:27.016270 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.018040 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.018047 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:27.018251 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:27.018287 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:33:27.018420 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 22:33:27.018601 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1211,"thread_start_us":656,"threads_started":1}
I20260806 22:33:27.020902 17160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4187f0b7e7d64839b926e2cc9e45741d" candidate_uuid: "bd2324b6b9e740919703b90b8d4fd476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abca76eee59f44919f14e34b4c5fb73c" is_pre_election: true
I20260806 22:33:27.021785 17160 raft_consensus.cc:2454] T 4187f0b7e7d64839b926e2cc9e45741d P abca76eee59f44919f14e34b4c5fb73c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd2324b6b9e740919703b90b8d4fd476 in term 0.
I20260806 22:33:27.022696 17249 leader_election.cc:302] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abca76eee59f44919f14e34b4c5fb73c, bd2324b6b9e740919703b90b8d4fd476; no voters:
I20260806 22:33:27.023316 17616 raft_consensus.cc:2781] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:33:27.023566 17607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:27.008825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:50525 (local address 127.15.198.131:43729)
0806 22:33:27.009126 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:27.009135 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:27.010638 (+ 1503us) server_negotiation.cc:407] Connection header received
0806 22:33:27.010987 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:27.011021 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:27.011145 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:27.011266 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:27.012533 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.014166 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.019898 (+ 5732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.021678 (+ 1780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.021862 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:27.021988 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:27.022861 (+ 873us) server_negotiation.cc:299] Negotiation successful
0806 22:33:27.023103 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 22:33:27.023617 17616 raft_consensus.cc:482] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:33:27.023878 17616 raft_consensus.cc:3037] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:27.024909 17620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:27.008602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:43729 (local address 127.15.198.130:50525)
0806 22:33:27.010465 (+ 1863us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:27.010493 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:27.010705 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:27.011307 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:27.011312 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:27.011343 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:27.012286 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:27.012304 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.015927 (+ 3623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.015934 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:27.019669 (+ 3735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:27.019688 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.022305 (+ 2617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.022315 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:27.022610 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:27.022653 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:33:27.023924 (+ 1271us) client_negotiation.cc:240] Negotiation successful
0806 22:33:27.024133 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1695,"mutex_wait_us":1,"thread_start_us":1096,"threads_started":1}
I20260806 22:33:27.026394 17521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4187f0b7e7d64839b926e2cc9e45741d" candidate_uuid: "bd2324b6b9e740919703b90b8d4fd476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c03eb14b353e4dbea18b72ec10bf0481" is_pre_election: true
I20260806 22:33:27.027383 17521 raft_consensus.cc:2454] T 4187f0b7e7d64839b926e2cc9e45741d P c03eb14b353e4dbea18b72ec10bf0481 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd2324b6b9e740919703b90b8d4fd476 in term 0.
I20260806 22:33:27.028893 17616 raft_consensus.cc:504] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abca76eee59f44919f14e34b4c5fb73c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 36213 } } peers { permanent_uuid: "c03eb14b353e4dbea18b72ec10bf0481" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43729 } } peers { permanent_uuid: "bd2324b6b9e740919703b90b8d4fd476" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45767 } }
I20260806 22:33:27.029749 17616 leader_election.cc:288] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 [CANDIDATE]: Term 1 election: Requested vote from peers abca76eee59f44919f14e34b4c5fb73c (127.15.198.129:36213), c03eb14b353e4dbea18b72ec10bf0481 (127.15.198.131:43729)
I20260806 22:33:27.030594 17521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4187f0b7e7d64839b926e2cc9e45741d" candidate_uuid: "bd2324b6b9e740919703b90b8d4fd476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c03eb14b353e4dbea18b72ec10bf0481"
I20260806 22:33:27.030764 17160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4187f0b7e7d64839b926e2cc9e45741d" candidate_uuid: "bd2324b6b9e740919703b90b8d4fd476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abca76eee59f44919f14e34b4c5fb73c"
I20260806 22:33:27.030905 17521 raft_consensus.cc:3037] T 4187f0b7e7d64839b926e2cc9e45741d P c03eb14b353e4dbea18b72ec10bf0481 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:27.031096 17160 raft_consensus.cc:3037] T 4187f0b7e7d64839b926e2cc9e45741d P abca76eee59f44919f14e34b4c5fb73c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:27.035825 17521 raft_consensus.cc:2454] T 4187f0b7e7d64839b926e2cc9e45741d P c03eb14b353e4dbea18b72ec10bf0481 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd2324b6b9e740919703b90b8d4fd476 in term 1.
I20260806 22:33:27.035825 17160 raft_consensus.cc:2454] T 4187f0b7e7d64839b926e2cc9e45741d P abca76eee59f44919f14e34b4c5fb73c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd2324b6b9e740919703b90b8d4fd476 in term 1.
I20260806 22:33:27.036931 17249 leader_election.cc:302] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abca76eee59f44919f14e34b4c5fb73c, bd2324b6b9e740919703b90b8d4fd476; no voters:
I20260806 22:33:27.037496 17616 raft_consensus.cc:2781] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:33:27.038218 17616 raft_consensus.cc:686] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 [term 1 LEADER]: Becoming Leader. State: Replica: bd2324b6b9e740919703b90b8d4fd476, State: Running, Role: LEADER
I20260806 22:33:27.039307 17616 consensus_queue.cc:237] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abca76eee59f44919f14e34b4c5fb73c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 36213 } } peers { permanent_uuid: "c03eb14b353e4dbea18b72ec10bf0481" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43729 } } peers { permanent_uuid: "bd2324b6b9e740919703b90b8d4fd476" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45767 } }
I20260806 22:33:27.046777 16964 catalog_manager.cc:5495] T 4187f0b7e7d64839b926e2cc9e45741d P bd2324b6b9e740919703b90b8d4fd476 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd2324b6b9e740919703b90b8d4fd476 (127.15.198.130). New cstate: current_term: 1 leader_uuid: "bd2324b6b9e740919703b90b8d4fd476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abca76eee59f44919f14e34b4c5fb73c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 36213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c03eb14b353e4dbea18b72ec10bf0481" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd2324b6b9e740919703b90b8d4fd476" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45767 } health_report { overall_health: HEALTHY } } }
Aug 06 22:33:27 dist-test-slave-h3ft krb5kdc[16896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
W20260806 22:33:27.080821 17217 tablet.cc:2369] T 4187f0b7e7d64839b926e2cc9e45741d P abca76eee59f44919f14e34b4c5fb73c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:33:27.081486 17604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:27.057531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35582 (local address 127.15.198.129:36213)
0806 22:33:27.057815 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:27.057820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:27.059266 (+ 1446us) server_negotiation.cc:407] Connection header received
0806 22:33:27.059500 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:27.059505 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:27.059597 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:27.059708 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:27.062365 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.063681 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.065477 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.066009 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.066134 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:27.072396 (+ 6262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:27.072429 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:27.072459 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:27.072587 (+ 128us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:27.076718 (+ 4131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:27.078042 (+ 1324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:27.078053 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:27.078056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:27.078281 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:27.078832 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:27.078837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:27.078839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:27.079649 (+ 810us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:27.080184 (+ 535us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:27.080582 (+ 398us) server_negotiation.cc:299] Negotiation successful
0806 22:33:27.080879 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
Aug 06 22:33:27 dist-test-slave-h3ft krb5kdc[16896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055607, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:33:27 dist-test-slave-h3ft krb5kdc[16896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055607, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:33:27.158586 17604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:27.137533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35596 (local address 127.15.198.129:36213)
0806 22:33:27.137805 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:27.137814 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:27.138381 (+ 567us) server_negotiation.cc:407] Connection header received
0806 22:33:27.138663 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:27.138669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:27.138761 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:27.138899 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:27.141008 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.142024 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.143614 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.144125 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.144318 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:27.150636 (+ 6318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:27.150643 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:27.150647 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:27.150714 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:27.154483 (+ 3769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:27.155576 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:27.155582 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:27.155584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:27.155877 (+ 293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:27.156500 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:27.156504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:27.156506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:27.157059 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:27.157510 (+ 451us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:27.157786 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 22:33:27.158110 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
Aug 06 22:33:27 dist-test-slave-h3ft krb5kdc[16896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:33:27 dist-test-slave-h3ft krb5kdc[16896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:33:27.233186 17604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:27.211039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35608 (local address 127.15.198.129:36213)
0806 22:33:27.211374 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:27.211384 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:27.211849 (+ 465us) server_negotiation.cc:407] Connection header received
0806 22:33:27.212098 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:27.212103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:27.212192 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:27.212284 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:27.214839 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.215909 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.218132 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.218648 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.218780 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:27.225080 (+ 6300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:27.225087 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:27.225091 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:27.225153 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:27.229043 (+ 3890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:27.230155 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:27.230172 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:27.230174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:27.230360 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:27.231098 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:27.231105 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:27.231108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:27.231601 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:27.231884 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:27.232351 (+ 467us) server_negotiation.cc:299] Negotiation successful
0806 22:33:27.232659 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260806 22:33:27.236312 17104 tablet_service.cc:2908] Rejecting scan request for tablet 4187f0b7e7d64839b926e2cc9e45741d: Uninitialized: clean time has not yet been initialized
I20260806 22:33:27.241966 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 17050
I20260806 22:33:27.261830 17604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:27.243405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35618 (local address 127.15.198.129:36213)
0806 22:33:27.243717 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:27.243724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:27.244083 (+ 359us) server_negotiation.cc:407] Connection header received
0806 22:33:27.244335 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:27.244341 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:27.244452 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:27.244566 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:27.246365 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.247412 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.249274 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.249764 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.249916 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:27.253877 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:27.253884 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:27.253888 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:27.253951 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:27.257554 (+ 3603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:27.258814 (+ 1260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:27.258820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:27.258823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:27.259128 (+ 305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:27.259671 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:27.259676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:27.259678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:27.260202 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:27.260603 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:27.261054 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 22:33:27.261375 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260806 22:33:27.263134 17206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:27.369807 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 17050
I20260806 22:33:27.403295 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 17225
Aug 06 22:33:27 dist-test-slave-h3ft krb5kdc[16896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.130@KRBTEST.COM
I20260806 22:33:27.424633 17638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:27.404562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49986 (local address 127.15.198.130:45767)
0806 22:33:27.405335 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:27.405342 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:27.405365 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:33:27.405446 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:27.405450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:27.405538 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:27.405664 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:27.407362 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.408243 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.409770 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.410348 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.410491 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:27.415958 (+ 5467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:27.416018 (+ 60us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:27.416052 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:27.416118 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:27.420546 (+ 4428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:27.421602 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:27.421612 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:27.421615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:27.421871 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:27.422508 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:27.422513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:27.422515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:27.423104 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:27.423521 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:27.423858 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 22:33:27.424225 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":469,"threads_started":1}
I20260806 22:33:27.425444 17394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:27.530555 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 17225
I20260806 22:33:27.565117 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 17409
Aug 06 22:33:27 dist-test-slave-h3ft krb5kdc[16896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.131@KRBTEST.COM
I20260806 22:33:27.584635 17640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:27.566278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42130 (local address 127.15.198.131:43729)
0806 22:33:27.566902 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:27.566908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:27.566927 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:33:27.567058 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:27.567063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:27.567187 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:27.567301 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:27.569175 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.570167 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.571972 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:27.572483 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:27.572650 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:27.578132 (+ 5482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:27.578157 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:27.578180 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:27.578234 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:27.581565 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:27.582395 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:27.582404 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:27.582406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:27.582571 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:27.582936 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:27.582938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:27.582940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:27.583475 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:27.583724 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:27.584054 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 22:33:27.584264 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":363,"threads_started":1}
I20260806 22:33:27.585486 17575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:27.685513 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 17409
I20260806 22:33:27.717093 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 16921
I20260806 22:33:27.718739 17007 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:27.846647 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 16921
2026-08-06T22:33:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (5546 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:33:27 dist-test-slave-h3ft krb5kdc[17647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:33:27 dist-test-slave-h3ft krb5kdc[17647](info): set up 2 sockets
Aug 06 22:33:27 dist-test-slave-h3ft krb5kdc[17647](info): commencing operation
krb5kdc: starting...
W20260806 22:33:29.194880 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:33:29 dist-test-slave-h3ft krb5kdc[17647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:33:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:33:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:33:29.460652 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:41829
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:35051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:41829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:29.877676 17665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:29.878232 17665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:29.878432 17665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:29.886799 17665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:33:29.887045 17665 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:33:29.887168 17665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:29.887301 17665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:29.887408 17665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:33:29.887544 17665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:33:29.899633 17665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:35051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:41829
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:41829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:29.901546 17665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:29.903375 17665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:29.919121 17679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:29.919214 17677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:29.919129 17676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:29.920187 17665 server_base.cc:1034] running on GCE node
I20260806 22:33:29.921346 17665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:29.923345 17665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:29.924616 17665 hybrid_clock.cc:648] HybridClock initialized: now 1786055609924535 us; error 48 us; skew 500 ppm
I20260806 22:33:29.925102 17665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:29 dist-test-slave-h3ft krb5kdc[17647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:29.929286 17665 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:33:29.931325 17665 webserver.cc:460] Webserver started at http://127.15.198.190:41851/ using document root <none> and password file <none>
I20260806 22:33:29.932279 17665 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:29.932384 17665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:29.932842 17665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:29.936764 17665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "a1c028194e9b4a54b9bcd78c2b567fe4"
format_stamp: "Formatted at 2026-08-06 22:33:29 on dist-test-slave-h3ft"
server_key: "0f4244f208fc2e866980966393cce473"
server_key_iv: "cf5ad5e46854747016568f6b451a97ee"
server_key_version: "encryptionkey@0"
I20260806 22:33:29.937736 17665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "a1c028194e9b4a54b9bcd78c2b567fe4"
format_stamp: "Formatted at 2026-08-06 22:33:29 on dist-test-slave-h3ft"
server_key: "0f4244f208fc2e866980966393cce473"
server_key_iv: "cf5ad5e46854747016568f6b451a97ee"
server_key_version: "encryptionkey@0"
I20260806 22:33:29.944484 17665 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260806 22:33:29.949177 17689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:29.951222 17665 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:33:29.951491 17665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "a1c028194e9b4a54b9bcd78c2b567fe4"
format_stamp: "Formatted at 2026-08-06 22:33:29 on dist-test-slave-h3ft"
server_key: "0f4244f208fc2e866980966393cce473"
server_key_iv: "cf5ad5e46854747016568f6b451a97ee"
server_key_version: "encryptionkey@0"
I20260806 22:33:29.951812 17665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:29.978307 17665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:29.984678 17665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:29.985059 17665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:30.015367 17665 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:41829
I20260806 22:33:30.015370 17766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:41829 every 8 connection(s)
I20260806 22:33:30.017397 17665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:33:30.022554 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 17665
I20260806 22:33:30.022915 17767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:30.022957 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:33:30.023928 16154 external_mini_cluster.cc:1428] Setting key 25686ed822d604ac43aabc49b9e6ce59
I20260806 22:33:30.040424 17767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4: Bootstrap starting.
I20260806 22:33:30.045145 17767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:30.047075 17767 log.cc:824] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4: Log is configured to *not* fsync() on all Append() calls
Aug 06 22:33:30 dist-test-slave-h3ft krb5kdc[17647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:30.052825 17767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4: No bootstrap required, opened a new log
I20260806 22:33:30.061759 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:30.027071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47668 (local address 127.15.198.190:41829)
0806 22:33:30.028291 (+ 1220us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:30.028307 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:30.028367 (+ 60us) server_negotiation.cc:407] Connection header received
0806 22:33:30.030485 (+ 2118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:30.030526 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:30.031201 (+ 675us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:30.031677 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:30.034032 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:30.035861 (+ 1829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:30.037953 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:30.038825 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:30.039053 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:30.051162 (+ 12109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:30.051178 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:30.051204 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:30.051263 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:30.055354 (+ 4091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:30.056723 (+ 1369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:30.056735 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:30.056745 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:30.057046 (+ 301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:30.057617 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:30.057622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:30.057624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:30.058560 (+ 936us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:30.058944 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:30.059592 (+ 648us) server_negotiation.cc:299] Negotiation successful
0806 22:33:30.060225 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":941,"thread_start_us":495,"threads_started":1}
I20260806 22:33:30.065088 17767 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c028194e9b4a54b9bcd78c2b567fe4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41829 } }
I20260806 22:33:30.065673 17767 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:30.065828 17767 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1c028194e9b4a54b9bcd78c2b567fe4, State: Initialized, Role: FOLLOWER
I20260806 22:33:30.066710 17767 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c028194e9b4a54b9bcd78c2b567fe4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41829 } }
I20260806 22:33:30.067140 17767 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:33:30.067329 17767 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:33:30.067559 17767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:30.070988 17767 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c028194e9b4a54b9bcd78c2b567fe4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41829 } }
I20260806 22:33:30.071849 17767 leader_election.cc:302] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1c028194e9b4a54b9bcd78c2b567fe4; no voters:
I20260806 22:33:30.072903 17767 leader_election.cc:288] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:33:30.073110 17772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:33:30.075136 17772 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [term 1 LEADER]: Becoming Leader. State: Replica: a1c028194e9b4a54b9bcd78c2b567fe4, State: Running, Role: LEADER
I20260806 22:33:30.076092 17772 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c028194e9b4a54b9bcd78c2b567fe4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41829 } }
I20260806 22:33:30.077093 17767 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:33:30.080328 17774 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1c028194e9b4a54b9bcd78c2b567fe4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c028194e9b4a54b9bcd78c2b567fe4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41829 } } }
I20260806 22:33:30.080328 17773 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1c028194e9b4a54b9bcd78c2b567fe4. Latest consensus state: current_term: 1 leader_uuid: "a1c028194e9b4a54b9bcd78c2b567fe4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c028194e9b4a54b9bcd78c2b567fe4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41829 } } }
I20260806 22:33:30.081122 17773 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:33:30.081122 17774 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:33:30.087328 17780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:33:30.094869 17780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:33:30.108229 17780 catalog_manager.cc:1347] Generated new cluster ID: 189434be662442db91f20cfacba719c2
I20260806 22:33:30.108435 17780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:33:30.133991 17780 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:33:30.136492 17780 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:33:30.147328 17780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4: Generated new TSK 0
I20260806 22:33:30.148687 17780 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
I20260806 22:33:30.236773 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:41829
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:35051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:31.024711 17798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:31.025153 17798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:31.025360 17798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:31.033715 17798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:31.033949 17798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:31.034127 17798 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:33:31.034286 17798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:33:31.046144 17798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:35051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.198.190:41829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:31.048215 17798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:31.050145 17798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:31.068004 17804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:31.068058 17806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:31.068236 17809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:31.069286 17798 server_base.cc:1034] running on GCE node
I20260806 22:33:31.070196 17798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:31.071552 17798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:31.072835 17798 hybrid_clock.cc:648] HybridClock initialized: now 1786055611072755 us; error 52 us; skew 500 ppm
I20260806 22:33:31.073246 17798 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:31 dist-test-slave-h3ft krb5kdc[17647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055611, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:31.077713 17798 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:33:31.079977 17798 webserver.cc:460] Webserver started at http://127.15.198.129:44231/ using document root <none> and password file <none>
I20260806 22:33:31.081012 17798 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:31.081177 17798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:31.081688 17798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:31.085722 17798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "9559fd469adc4ab090c9c9b63e40e889"
format_stamp: "Formatted at 2026-08-06 22:33:31 on dist-test-slave-h3ft"
server_key: "cd47dc4180f93cc4726ec28208677033"
server_key_iv: "1a46064a38a4f8c5e1100f80df3afd5b"
server_key_version: "encryptionkey@0"
I20260806 22:33:31.086889 17798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "9559fd469adc4ab090c9c9b63e40e889"
format_stamp: "Formatted at 2026-08-06 22:33:31 on dist-test-slave-h3ft"
server_key: "cd47dc4180f93cc4726ec28208677033"
server_key_iv: "1a46064a38a4f8c5e1100f80df3afd5b"
server_key_version: "encryptionkey@0"
I20260806 22:33:31.094377 17798 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 22:33:31.099769 17816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:31.101852 17798 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260806 22:33:31.102145 17798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "9559fd469adc4ab090c9c9b63e40e889"
format_stamp: "Formatted at 2026-08-06 22:33:31 on dist-test-slave-h3ft"
server_key: "cd47dc4180f93cc4726ec28208677033"
server_key_iv: "1a46064a38a4f8c5e1100f80df3afd5b"
server_key_version: "encryptionkey@0"
I20260806 22:33:31.102463 17798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:31.123874 17798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:31.129745 17798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:31.130103 17798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:31.133778 17798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:31.133910 17798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:31.134101 17798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:31.134194 17798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:33:31 dist-test-slave-h3ft krb5kdc[17647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055611, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:31.166977 17833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:31.135450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:38831 (local address 127.15.198.190:41829)
0806 22:33:31.136930 (+ 1480us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:31.136940 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:31.137723 (+ 783us) server_negotiation.cc:407] Connection header received
0806 22:33:31.140367 (+ 2644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:31.140374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:31.140539 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:31.140747 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:31.142812 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:31.144604 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:31.146610 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:31.147539 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:31.147729 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:31.155119 (+ 7390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:31.155133 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:31.155141 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:31.155224 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:31.160290 (+ 5066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:31.161991 (+ 1701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:31.162002 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:31.162007 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:31.162241 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:31.163237 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:31.163247 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:31.163254 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:31.163942 (+ 688us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:31.164163 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:31.165789 (+ 1626us) server_negotiation.cc:299] Negotiation successful
0806 22:33:31.166215 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1294,"thread_start_us":1137,"threads_started":1}
I20260806 22:33:31.169099 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:31.136042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41829 (local address 127.15.198.129:38831)
0806 22:33:31.137483 (+ 1441us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:31.137540 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:31.139885 (+ 2345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:31.140960 (+ 1075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:31.140969 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:31.141842 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:31.142584 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:31.142603 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:31.144836 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:31.144845 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:31.146438 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:31.146451 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:31.147812 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:31.147822 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:31.148247 (+ 425us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:31.149342 (+ 1095us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:31.149392 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:31.154587 (+ 5195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:31.160568 (+ 5981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:31.160594 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:31.160636 (+ 42us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:31.161796 (+ 1160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:31.162446 (+ 650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:31.162461 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:31.162469 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:31.162973 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:31.164167 (+ 1194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:31.164182 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:31.165047 (+ 865us) client_negotiation.cc:769] Sending connection context
0806 22:33:31.165606 (+ 559us) client_negotiation.cc:240] Negotiation successful
0806 22:33:31.166548 (+ 942us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1059,"thread_start_us":748,"threads_started":1}
I20260806 22:33:31.188715 17798 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:44893
I20260806 22:33:31.188735 17973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:44893 every 8 connection(s)
I20260806 22:33:31.191444 17798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:33:31.194697 17976 heartbeater.cc:347] Connected to a master server at 127.15.198.190:41829
I20260806 22:33:31.195034 17976 heartbeater.cc:464] Registering TS with master...
I20260806 22:33:31.196072 17976 heartbeater.cc:511] Master 127.15.198.190:41829 requested a full tablet report, sending...
I20260806 22:33:31.197045 17833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:31.178813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:36307 (local address 127.15.198.190:41829)
0806 22:33:31.179110 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:31.179116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:31.180455 (+ 1339us) server_negotiation.cc:407] Connection header received
0806 22:33:31.180715 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:31.180722 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:31.180818 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:31.180931 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:31.182707 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:31.183698 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:31.185868 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:31.186372 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:31.186563 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:31.189419 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:31.189428 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:31.189433 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:31.189484 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:31.193718 (+ 4234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:31.194643 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:31.194648 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:31.194651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:31.194874 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:31.195418 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:31.195422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:31.195424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:31.195811 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:31.195968 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:31.196513 (+ 545us) server_negotiation.cc:299] Negotiation successful
0806 22:33:31.196740 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 22:33:31.197327 17950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:31.178906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41829 (local address 127.15.198.129:36307)
0806 22:33:31.180310 (+ 1404us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:31.180342 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:31.180562 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:31.180966 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:31.180970 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:31.181856 (+ 886us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:31.182528 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:31.182554 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:31.183858 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:31.183864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:31.185700 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:31.185714 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:31.186587 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:31.186594 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:31.186773 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:31.187732 (+ 959us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:31.187752 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:31.189179 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:31.193871 (+ 4692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:31.193879 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:31.193886 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:31.194528 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:31.195043 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:31.195051 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:31.195057 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:31.195260 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:31.195958 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:31.195963 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:31.196217 (+ 254us) client_negotiation.cc:769] Sending connection context
0806 22:33:31.196383 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 22:33:31.196797 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1209,"thread_start_us":972,"threads_started":1}
I20260806 22:33:31.199370 17709 ts_manager.cc:194] Registered new tserver with Master: 9559fd469adc4ab090c9c9b63e40e889 (127.15.198.129:44893)
I20260806 22:33:31.199491 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 17798
I20260806 22:33:31.199869 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:33:31.200907 16154 external_mini_cluster.cc:1428] Setting key e76df66baad316ee5844e8a8224d5a19
I20260806 22:33:31.201675 17709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:38831
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
I20260806 22:33:31.264262 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:41829
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:35051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:31.699767 17980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:31.700167 17980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:31.700394 17980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:31.709190 17980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:31.709352 17980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:31.709522 17980 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:33:31.709633 17980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:33:31.721257 17980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:35051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.198.190:41829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:31.723608 17980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:31.725672 17980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:31.743990 17988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:31.744042 17989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:31.744055 17993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:31.744503 17980 server_base.cc:1034] running on GCE node
I20260806 22:33:31.745745 17980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:31.747046 17980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:31.748344 17980 hybrid_clock.cc:648] HybridClock initialized: now 1786055611748264 us; error 44 us; skew 500 ppm
I20260806 22:33:31.748744 17980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:31 dist-test-slave-h3ft krb5kdc[17647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055611, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:31.753278 17980 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:33:31.755715 17980 webserver.cc:460] Webserver started at http://127.15.198.130:41193/ using document root <none> and password file <none>
I20260806 22:33:31.756903 17980 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:31.757031 17980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:31.757465 17980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:31.761835 17980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "a738f94a6da44348a4d1f0cc0b8105d3"
format_stamp: "Formatted at 2026-08-06 22:33:31 on dist-test-slave-h3ft"
server_key: "2472b1d49fc17c901d3a6f4f06530b5a"
server_key_iv: "af304097d646118bd41a064e1ba3cbe7"
server_key_version: "encryptionkey@0"
I20260806 22:33:31.763113 17980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "a738f94a6da44348a4d1f0cc0b8105d3"
format_stamp: "Formatted at 2026-08-06 22:33:31 on dist-test-slave-h3ft"
server_key: "2472b1d49fc17c901d3a6f4f06530b5a"
server_key_iv: "af304097d646118bd41a064e1ba3cbe7"
server_key_version: "encryptionkey@0"
I20260806 22:33:31.770821 17980 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260806 22:33:31.775997 18001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:31.778083 17980 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:33:31.778339 17980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "a738f94a6da44348a4d1f0cc0b8105d3"
format_stamp: "Formatted at 2026-08-06 22:33:31 on dist-test-slave-h3ft"
server_key: "2472b1d49fc17c901d3a6f4f06530b5a"
server_key_iv: "af304097d646118bd41a064e1ba3cbe7"
server_key_version: "encryptionkey@0"
I20260806 22:33:31.778671 17980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:31.796623 17980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:31.802939 17980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:31.803417 17980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:31.807500 17980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:31.807657 17980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:31.807848 17980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:31.807963 17980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:33:31 dist-test-slave-h3ft krb5kdc[17647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055611, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:31.841163 18013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:31.808447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:33843 (local address 127.15.198.190:41829)
0806 22:33:31.809519 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:31.809528 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:31.810889 (+ 1361us) server_negotiation.cc:407] Connection header received
0806 22:33:31.814829 (+ 3940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:31.814841 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:31.814980 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:31.815159 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:31.819047 (+ 3888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:31.820164 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:31.824152 (+ 3988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:31.824937 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:31.825119 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:31.831423 (+ 6304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:31.831434 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:31.831440 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:31.831517 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:31.836057 (+ 4540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:31.837566 (+ 1509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:31.837577 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:31.837584 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:31.837791 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:31.838339 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:31.838346 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:31.838350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:31.838939 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:31.839257 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:31.840398 (+ 1141us) server_negotiation.cc:299] Negotiation successful
0806 22:33:31.840696 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":891,"thread_start_us":680,"threads_started":1}
I20260806 22:33:31.843233 18015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:31.809352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41829 (local address 127.15.198.130:33843)
0806 22:33:31.810613 (+ 1261us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:31.810700 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:31.814239 (+ 3539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:31.815635 (+ 1396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:31.815660 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:31.817027 (+ 1367us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:31.818673 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:31.818715 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:31.820379 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:31.820389 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:31.823856 (+ 3467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:31.823889 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:31.825309 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:31.825319 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:31.825816 (+ 497us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:31.827246 (+ 1430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:31.827297 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:31.830927 (+ 3630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:31.836287 (+ 5360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:31.836305 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:31.836339 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:31.837335 (+ 996us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:31.837941 (+ 606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:31.837945 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:31.837947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:31.838186 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:31.839238 (+ 1052us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:31.839248 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:31.839927 (+ 679us) client_negotiation.cc:769] Sending connection context
0806 22:33:31.840264 (+ 337us) client_negotiation.cc:240] Negotiation successful
0806 22:33:31.840983 (+ 719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"thread_start_us":506,"threads_started":1}
I20260806 22:33:31.872713 18013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:31.853768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:46873 (local address 127.15.198.190:41829)
0806 22:33:31.854104 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:31.854117 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:31.854872 (+ 755us) server_negotiation.cc:407] Connection header received
0806 22:33:31.855222 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:31.855232 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:31.855379 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:31.855526 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:31.857799 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:31.858788 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:31.860918 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:31.862037 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:31.862249 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:31.864799 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:31.864811 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:31.864821 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:31.864892 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:31.869223 (+ 4331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:31.870167 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:31.870171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:31.870174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:31.870327 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:31.870762 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:31.870774 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:31.870776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:31.871285 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:31.871480 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:31.871987 (+ 507us) server_negotiation.cc:299] Negotiation successful
0806 22:33:31.872268 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 22:33:31.873421 18107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:31.853207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41829 (local address 127.15.198.130:46873)
0806 22:33:31.854656 (+ 1449us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:31.854689 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:31.854986 (+ 297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:31.855748 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:31.855752 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:31.856952 (+ 1200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:31.857588 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:31.857607 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:31.858975 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:31.858981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:31.860705 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:31.860729 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:31.862289 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:31.862294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:31.862435 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:31.863488 (+ 1053us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:31.863511 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:31.864498 (+ 987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:31.869435 (+ 4937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:31.869442 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:31.869447 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:31.869983 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:31.870465 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:31.870471 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:31.870476 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:31.870636 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:31.871422 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:31.871432 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:31.871681 (+ 249us) client_negotiation.cc:769] Sending connection context
0806 22:33:31.871869 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 22:33:31.872414 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1189,"thread_start_us":945,"threads_started":1}
I20260806 22:33:31.885673 17980 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:43195
I20260806 22:33:31.885691 18159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:43195 every 8 connection(s)
I20260806 22:33:31.887517 17980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:33:31.890383 18160 heartbeater.cc:347] Connected to a master server at 127.15.198.190:41829
I20260806 22:33:31.890631 18160 heartbeater.cc:464] Registering TS with master...
I20260806 22:33:31.891530 18160 heartbeater.cc:511] Master 127.15.198.190:41829 requested a full tablet report, sending...
I20260806 22:33:31.893340 17707 ts_manager.cc:194] Registered new tserver with Master: a738f94a6da44348a4d1f0cc0b8105d3 (127.15.198.130:43195)
I20260806 22:33:31.894543 17707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:33843
I20260806 22:33:31.896878 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 17980
I20260806 22:33:31.897174 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:33:31.897748 16154 external_mini_cluster.cc:1428] Setting key 0e589bfeb5eb56ba371045652c792170
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
I20260806 22:33:31.965181 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:41829
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:35051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:33:32.205711 17976 heartbeater.cc:503] Master 127.15.198.190:41829 was elected leader, sending a full tablet report...
W20260806 22:33:32.420094 18165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:32.420614 18165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:32.420826 18165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:32.430519 18165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:32.430761 18165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:32.430964 18165 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:33:32.431165 18165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:33:32.443617 18165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:35051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.198.190:41829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:32.445521 18165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:32.447536 18165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:32.465487 18174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:32.465531 18171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:32.466099 18170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:32.466946 18165 server_base.cc:1034] running on GCE node
I20260806 22:33:32.467859 18165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:32.469229 18165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:32.470543 18165 hybrid_clock.cc:648] HybridClock initialized: now 1786055612470426 us; error 75 us; skew 500 ppm
I20260806 22:33:32.470974 18165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:32 dist-test-slave-h3ft krb5kdc[17647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055612, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:32.475708 18165 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:33:32.477927 18165 webserver.cc:460] Webserver started at http://127.15.198.131:43465/ using document root <none> and password file <none>
I20260806 22:33:32.479132 18165 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:32.479305 18165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:32.479786 18165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:32.483708 18165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "8478adc38a7241bd8b78f9b9bd6974d2"
format_stamp: "Formatted at 2026-08-06 22:33:32 on dist-test-slave-h3ft"
server_key: "a2727cef57a2f436ca4e8f766e77cb2c"
server_key_iv: "3eebfb92f1a2d3e9999085a87e259265"
server_key_version: "encryptionkey@0"
I20260806 22:33:32.484920 18165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "8478adc38a7241bd8b78f9b9bd6974d2"
format_stamp: "Formatted at 2026-08-06 22:33:32 on dist-test-slave-h3ft"
server_key: "a2727cef57a2f436ca4e8f766e77cb2c"
server_key_iv: "3eebfb92f1a2d3e9999085a87e259265"
server_key_version: "encryptionkey@0"
I20260806 22:33:32.492306 18165 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 22:33:32.497386 18186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:32.499539 18165 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260806 22:33:32.499763 18165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "8478adc38a7241bd8b78f9b9bd6974d2"
format_stamp: "Formatted at 2026-08-06 22:33:32 on dist-test-slave-h3ft"
server_key: "a2727cef57a2f436ca4e8f766e77cb2c"
server_key_iv: "3eebfb92f1a2d3e9999085a87e259265"
server_key_version: "encryptionkey@0"
I20260806 22:33:32.500061 18165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:32.517473 18165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:32.523178 18165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:32.523506 18165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:32.527037 18165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:32.527311 18165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 22:33:32.527562 18165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:32.527653 18165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:33:32 dist-test-slave-h3ft krb5kdc[17647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055612, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:32.561137 18196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:32.528634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:33883 (local address 127.15.198.190:41829)
0806 22:33:32.529305 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:32.529315 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:32.531365 (+ 2050us) server_negotiation.cc:407] Connection header received
0806 22:33:32.534402 (+ 3037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:32.534423 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:32.534568 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:32.534746 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:32.537870 (+ 3124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.539321 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.542972 (+ 3651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.543733 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.543912 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:32.550673 (+ 6761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:32.550681 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:32.550686 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:32.550734 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:32.555692 (+ 4958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:32.557223 (+ 1531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:32.557230 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:32.557233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:32.557402 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:32.558296 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:32.558301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:32.558304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:32.558728 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:32.558866 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:32.560478 (+ 1612us) server_negotiation.cc:299] Negotiation successful
0806 22:33:32.560734 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":341,"threads_started":1}
I20260806 22:33:32.563966 18198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:32.529785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41829 (local address 127.15.198.131:33883)
0806 22:33:32.531130 (+ 1345us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:32.531191 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:32.533904 (+ 2713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:32.535253 (+ 1349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:32.535263 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:32.536335 (+ 1072us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:32.537601 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:32.537639 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.539534 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.539543 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:32.542746 (+ 3203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:32.542763 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.544077 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.544088 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:32.544388 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:32.545547 (+ 1159us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:32.545578 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:32.550288 (+ 4710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:32.555931 (+ 5643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:32.555950 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:32.555986 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:32.557011 (+ 1025us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:32.557554 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:32.557562 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:32.557568 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:32.558010 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:32.558910 (+ 900us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:32.558923 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:32.559819 (+ 896us) client_negotiation.cc:769] Sending connection context
0806 22:33:32.560312 (+ 493us) client_negotiation.cc:240] Negotiation successful
0806 22:33:32.561245 (+ 933us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":905,"thread_start_us":583,"threads_started":1}
I20260806 22:33:32.596592 18165 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:41409
I20260806 22:33:32.596661 18352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:41409 every 8 connection(s)
I20260806 22:33:32.597460 18196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:32.577292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:57377 (local address 127.15.198.190:41829)
0806 22:33:32.577567 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:32.577575 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:32.579134 (+ 1559us) server_negotiation.cc:407] Connection header received
0806 22:33:32.579675 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:32.579693 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:32.579798 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:32.579885 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:32.581749 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.583087 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.585059 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.585765 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.585956 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:32.589552 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:32.589563 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:32.589569 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:32.589632 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:32.593712 (+ 4080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:32.594648 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:32.594662 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:32.594666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:32.594835 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:32.595381 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:32.595394 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:32.595398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:32.595789 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:32.595991 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:32.596709 (+ 718us) server_negotiation.cc:299] Negotiation successful
0806 22:33:32.597055 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 22:33:32.597508 18314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:32.577466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41829 (local address 127.15.198.131:57377)
0806 22:33:32.578895 (+ 1429us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:32.578923 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:32.579390 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:32.580249 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:32.580257 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:32.581071 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:32.581535 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:32.581548 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.583413 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.583419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:32.584873 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:32.584890 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.586112 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.586123 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:32.586442 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:32.587612 (+ 1170us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:32.587635 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:32.589269 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:32.593933 (+ 4664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:32.593941 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:32.593946 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:32.594482 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:32.595017 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:32.595026 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:32.595031 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:32.595213 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:32.595938 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:32.595943 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:32.596290 (+ 347us) client_negotiation.cc:769] Sending connection context
0806 22:33:32.596526 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 22:33:32.597089 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1224,"thread_start_us":491,"threads_started":1}
I20260806 22:33:32.598791 18165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:33:32.600832 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 18165
I20260806 22:33:32.601133 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:33:32.601790 16154 external_mini_cluster.cc:1428] Setting key 885856c57d88de1ce064a55c445de106
I20260806 22:33:32.604205 18353 heartbeater.cc:347] Connected to a master server at 127.15.198.190:41829
I20260806 22:33:32.604576 18353 heartbeater.cc:464] Registering TS with master...
I20260806 22:33:32.605757 18353 heartbeater.cc:511] Master 127.15.198.190:41829 requested a full tablet report, sending...
I20260806 22:33:32.607898 17709 ts_manager.cc:194] Registered new tserver with Master: 8478adc38a7241bd8b78f9b9bd6974d2 (127.15.198.131:41409)
I20260806 22:33:32.608978 17709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:33883
I20260806 22:33:32.620971 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:33:32 dist-test-slave-h3ft krb5kdc[17647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055612, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:33:32 dist-test-slave-h3ft krb5kdc[17647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055612, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:32.695252 18196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:32.674892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47680 (local address 127.15.198.190:41829)
0806 22:33:32.675273 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:32.675282 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:32.675575 (+ 293us) server_negotiation.cc:407] Connection header received
0806 22:33:32.675780 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:32.675785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:32.675939 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:32.676048 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:32.678023 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.679345 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.681200 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.681745 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.681905 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:32.687979 (+ 6074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:32.687986 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:32.687990 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:32.688099 (+ 109us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:32.691591 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:32.692617 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:32.692623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:32.692626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:32.692799 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:32.693295 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:32.693300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:32.693304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:32.693698 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:32.694054 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:32.694435 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 22:33:32.694760 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185}
I20260806 22:33:32.701833 17709 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:33:32.712455 17709 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:33:32.753753 18379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:32.734250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45830 (local address 127.15.198.131:41409)
0806 22:33:32.741684 (+ 7434us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:32.741695 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:32.742410 (+ 715us) server_negotiation.cc:407] Connection header received
0806 22:33:32.742716 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:32.742723 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:32.742938 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:32.743204 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:32.744599 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.746434 (+ 1835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.749954 (+ 3520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.752007 (+ 2053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.752278 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:32.752611 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:32.752977 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 22:33:32.753223 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7168,"thread_start_us":7273,"threads_started":1}
I20260806 22:33:32.753832 18371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:32.732020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:44893 (local address 127.0.0.1:52300)
0806 22:33:32.733073 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:32.733129 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:32.733366 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:32.743282 (+ 9916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:32.743295 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:32.743332 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:32.744060 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:32.744079 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.746685 (+ 2606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.746694 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:32.749790 (+ 3096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:32.749855 (+ 65us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.752334 (+ 2479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.752341 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:32.752655 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:32.752726 (+ 71us) client_negotiation.cc:769] Sending connection context
0806 22:33:32.752893 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 22:33:32.753224 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"mutex_wait_us":149,"thread_start_us":530,"threads_started":1}
I20260806 22:33:32.754532 18375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:32.733653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:41409 (local address 127.0.0.1:45830)
0806 22:33:32.742259 (+ 8606us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:32.742278 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:32.742528 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:32.743718 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:32.743728 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:32.743760 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:32.744368 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:32.744378 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.746668 (+ 2290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.746673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:32.749696 (+ 3023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:32.749729 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.752283 (+ 2554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.752289 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:32.752651 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:32.752727 (+ 76us) client_negotiation.cc:769] Sending connection context
0806 22:33:32.752885 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 22:33:32.753240 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8476,"mutex_wait_us":236,"thread_start_us":7140,"threads_started":1}
I20260806 22:33:32.756000 18374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:32.732645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52300 (local address 127.15.198.129:44893)
0806 22:33:32.742350 (+ 9705us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:32.742360 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:32.742390 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:33:32.742540 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:32.742550 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:32.742939 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:32.743821 (+ 882us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:32.744292 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.746374 (+ 2082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.750099 (+ 3725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.752008 (+ 1909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.754298 (+ 2290us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:32.754622 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:32.754827 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 22:33:32.755253 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9501,"thread_start_us":9685,"threads_started":1}
I20260806 22:33:32.756846 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:32.732032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:43195 (local address 127.0.0.1:54008)
0806 22:33:32.741192 (+ 9160us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:32.741221 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:32.741441 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:32.746034 (+ 4593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:32.746040 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:32.746073 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:32.746764 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:32.746780 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.749257 (+ 2477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.749263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:32.751512 (+ 2249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:32.751529 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.753823 (+ 2294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.753833 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:32.756199 (+ 2366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:32.756246 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 22:33:32.756350 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 22:33:32.756532 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8954,"mutex_wait_us":889,"thread_start_us":9683,"threads_started":1}
I20260806 22:33:32.757715 18372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:32.732277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54008 (local address 127.15.198.130:43195)
0806 22:33:32.745078 (+ 12801us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:32.745090 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:32.745131 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:33:32.745257 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:32.745268 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:32.745620 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:32.745883 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:32.747172 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.749055 (+ 1883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.751738 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:32.753196 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:32.753405 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:32.753615 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:32.757185 (+ 3570us) server_negotiation.cc:299] Negotiation successful
0806 22:33:32.757386 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12596,"thread_start_us":12373,"threads_started":1}
I20260806 22:33:32.761435 18251 tablet_service.cc:1467] Processing CreateTablet for tablet 56972b8b8f4c4566a00b4cd811c2b2f1 (DEFAULT_TABLE table=test-table [id=e67f843f35b546f88dbfd9eadff79d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:32.761415 18072 tablet_service.cc:1467] Processing CreateTablet for tablet 56972b8b8f4c4566a00b4cd811c2b2f1 (DEFAULT_TABLE table=test-table [id=e67f843f35b546f88dbfd9eadff79d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:32.761421 17890 tablet_service.cc:1467] Processing CreateTablet for tablet 56972b8b8f4c4566a00b4cd811c2b2f1 (DEFAULT_TABLE table=test-table [id=e67f843f35b546f88dbfd9eadff79d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:32.763334 18251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56972b8b8f4c4566a00b4cd811c2b2f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:32.763345 18072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56972b8b8f4c4566a00b4cd811c2b2f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:32.763334 17890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56972b8b8f4c4566a00b4cd811c2b2f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:32.780228 18380 tablet_bootstrap.cc:492] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3: Bootstrap starting.
I20260806 22:33:32.786062 18380 tablet_bootstrap.cc:654] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:32.787925 18382 tablet_bootstrap.cc:492] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2: Bootstrap starting.
I20260806 22:33:32.788635 18380 log.cc:824] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:33:32.790952 18383 tablet_bootstrap.cc:492] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889: Bootstrap starting.
I20260806 22:33:32.795804 18382 tablet_bootstrap.cc:654] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:32.797461 18380 tablet_bootstrap.cc:492] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3: No bootstrap required, opened a new log
I20260806 22:33:32.797461 18383 tablet_bootstrap.cc:654] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:32.797727 18382 log.cc:824] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:33:32.797998 18380 ts_tablet_manager.cc:1397] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.004s
I20260806 22:33:32.799590 18383 log.cc:824] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889: Log is configured to *not* fsync() on all Append() calls
I20260806 22:33:32.802922 18382 tablet_bootstrap.cc:492] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2: No bootstrap required, opened a new log
I20260806 22:33:32.803642 18382 ts_tablet_manager.cc:1397] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.009s
I20260806 22:33:32.804569 18383 tablet_bootstrap.cc:492] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889: No bootstrap required, opened a new log
I20260806 22:33:32.804980 18383 ts_tablet_manager.cc:1397] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260806 22:33:32.814873 18382 raft_consensus.cc:348] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:32.814903 18380 raft_consensus.cc:348] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:32.815624 18380 raft_consensus.cc:374] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:32.815653 18382 raft_consensus.cc:374] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:32.815776 18382 raft_consensus.cc:729] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8478adc38a7241bd8b78f9b9bd6974d2, State: Initialized, Role: FOLLOWER
I20260806 22:33:32.815848 18380 raft_consensus.cc:729] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a738f94a6da44348a4d1f0cc0b8105d3, State: Initialized, Role: FOLLOWER
I20260806 22:33:32.816746 18382 consensus_queue.cc:260] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:32.816884 18380 consensus_queue.cc:260] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:32.819237 18353 heartbeater.cc:503] Master 127.15.198.190:41829 was elected leader, sending a full tablet report...
I20260806 22:33:32.820981 18383 raft_consensus.cc:348] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:32.821552 18382 ts_tablet_manager.cc:1428] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2: Time spent starting tablet: real 0.018s user 0.009s sys 0.006s
I20260806 22:33:32.821838 18383 raft_consensus.cc:374] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:32.822005 18383 raft_consensus.cc:729] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9559fd469adc4ab090c9c9b63e40e889, State: Initialized, Role: FOLLOWER
I20260806 22:33:32.823119 18160 heartbeater.cc:503] Master 127.15.198.190:41829 was elected leader, sending a full tablet report...
I20260806 22:33:32.823078 18383 consensus_queue.cc:260] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:32.824452 18380 ts_tablet_manager.cc:1428] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3: Time spent starting tablet: real 0.026s user 0.014s sys 0.011s
I20260806 22:33:32.826817 18383 ts_tablet_manager.cc:1428] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889: Time spent starting tablet: real 0.022s user 0.010s sys 0.012s
W20260806 22:33:32.858671 18354 tablet.cc:2369] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:33:32.890535 18161 tablet.cc:2369] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:33:32.945359 17977 tablet.cc:2369] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:33:33.024869 18394 raft_consensus.cc:482] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:33:33.025460 18394 raft_consensus.cc:504] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.028400 18394 leader_election.cc:288] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a738f94a6da44348a4d1f0cc0b8105d3 (127.15.198.130:43195), 8478adc38a7241bd8b78f9b9bd6974d2 (127.15.198.131:41409)
I20260806 22:33:33.039873 18372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:33.029072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:59325 (local address 127.15.198.130:43195)
0806 22:33:33.029325 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:33.029332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:33.030999 (+ 1667us) server_negotiation.cc:407] Connection header received
0806 22:33:33.031538 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:33.031544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:33.031631 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:33.031704 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:33.033292 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.034245 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.037395 (+ 3150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.038569 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.038762 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:33.038854 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:33.039328 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 22:33:33.039552 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 22:33:33.039907 18379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:33.029102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:42099 (local address 127.15.198.131:41409)
0806 22:33:33.029325 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:33.029332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:33.031381 (+ 2049us) server_negotiation.cc:407] Connection header received
0806 22:33:33.031528 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:33.031534 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:33.031632 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:33.031732 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:33.033016 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.034211 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.037396 (+ 3185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.038367 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.038546 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:33.038631 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:33.039158 (+ 527us) server_negotiation.cc:299] Negotiation successful
0806 22:33:33.039345 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 22:33:33.040235 18396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:33.029550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:41409 (local address 127.15.198.129:42099)
0806 22:33:33.031224 (+ 1674us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:33.031243 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:33.031386 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:33.031779 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:33.031784 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:33.031820 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:33.032686 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:33.032705 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.034415 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.034423 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:33.037162 (+ 2739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:33.037180 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.038648 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.038657 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:33.038856 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:33.038938 (+ 82us) client_negotiation.cc:769] Sending connection context
0806 22:33:33.039096 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 22:33:33.039261 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1538,"mutex_wait_us":82,"thread_start_us":718,"threads_started":1}
I20260806 22:33:33.040302 18395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:33.029362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:43195 (local address 127.15.198.129:59325)
0806 22:33:33.030798 (+ 1436us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:33.030823 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:33.031249 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:33.031954 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:33.031957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:33.031996 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:33.033058 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:33.033075 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.034450 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.034454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:33.037164 (+ 2710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:33.037180 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.038852 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.038857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:33.039133 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:33.039169 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:33:33.039265 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 22:33:33.039406 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1187,"mutex_wait_us":97,"thread_start_us":1391,"threads_started":1}
I20260806 22:33:33.042086 18284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56972b8b8f4c4566a00b4cd811c2b2f1" candidate_uuid: "9559fd469adc4ab090c9c9b63e40e889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" is_pre_election: true
I20260806 22:33:33.041982 18101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56972b8b8f4c4566a00b4cd811c2b2f1" candidate_uuid: "9559fd469adc4ab090c9c9b63e40e889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" is_pre_election: true
I20260806 22:33:33.043182 18284 raft_consensus.cc:2454] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9559fd469adc4ab090c9c9b63e40e889 in term 0.
I20260806 22:33:33.043182 18101 raft_consensus.cc:2454] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9559fd469adc4ab090c9c9b63e40e889 in term 0.
I20260806 22:33:33.044246 17818 leader_election.cc:302] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8478adc38a7241bd8b78f9b9bd6974d2, 9559fd469adc4ab090c9c9b63e40e889, a738f94a6da44348a4d1f0cc0b8105d3; no voters:
I20260806 22:33:33.044880 18394 raft_consensus.cc:2781] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:33:33.045197 18394 raft_consensus.cc:482] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:33:33.045392 18394 raft_consensus.cc:3037] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:33.049194 18394 raft_consensus.cc:504] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.050053 18394 leader_election.cc:288] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [CANDIDATE]: Term 1 election: Requested vote from peers a738f94a6da44348a4d1f0cc0b8105d3 (127.15.198.130:43195), 8478adc38a7241bd8b78f9b9bd6974d2 (127.15.198.131:41409)
I20260806 22:33:33.050899 18284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56972b8b8f4c4566a00b4cd811c2b2f1" candidate_uuid: "9559fd469adc4ab090c9c9b63e40e889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8478adc38a7241bd8b78f9b9bd6974d2"
I20260806 22:33:33.050894 18101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56972b8b8f4c4566a00b4cd811c2b2f1" candidate_uuid: "9559fd469adc4ab090c9c9b63e40e889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a738f94a6da44348a4d1f0cc0b8105d3"
I20260806 22:33:33.051347 18284 raft_consensus.cc:3037] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:33.051390 18101 raft_consensus.cc:3037] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:33.054755 18101 raft_consensus.cc:2454] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9559fd469adc4ab090c9c9b63e40e889 in term 1.
I20260806 22:33:33.054843 18284 raft_consensus.cc:2454] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9559fd469adc4ab090c9c9b63e40e889 in term 1.
I20260806 22:33:33.055642 17817 leader_election.cc:302] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8478adc38a7241bd8b78f9b9bd6974d2, 9559fd469adc4ab090c9c9b63e40e889, a738f94a6da44348a4d1f0cc0b8105d3; no voters:
I20260806 22:33:33.056277 18394 raft_consensus.cc:2781] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:33:33.057108 18394 raft_consensus.cc:686] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [term 1 LEADER]: Becoming Leader. State: Replica: 9559fd469adc4ab090c9c9b63e40e889, State: Running, Role: LEADER
I20260806 22:33:33.057941 18394 consensus_queue.cc:237] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.065276 17705 catalog_manager.cc:5495] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9559fd469adc4ab090c9c9b63e40e889 (127.15.198.129). New cstate: current_term: 1 leader_uuid: "9559fd469adc4ab090c9c9b63e40e889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } health_report { overall_health: UNKNOWN } } }
I20260806 22:33:33.159461 18374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:33.151824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52308 (local address 127.15.198.129:44893)
0806 22:33:33.152141 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:33.152150 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:33.152340 (+ 190us) server_negotiation.cc:407] Connection header received
0806 22:33:33.152900 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:33.152906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:33.153031 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:33.153209 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:33.153702 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.154646 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.156290 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.156850 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.157015 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:33.157755 (+ 740us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:33.158633 (+ 878us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:33.158850 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:33.158965 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 22:33:33.159108 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 22:33:33.173620 18101 raft_consensus.cc:1261] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3 [term 1 FOLLOWER]: Refusing update from remote peer 9559fd469adc4ab090c9c9b63e40e889: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:33:33.173666 18284 raft_consensus.cc:1261] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2 [term 1 FOLLOWER]: Refusing update from remote peer 9559fd469adc4ab090c9c9b63e40e889: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:33:33.175513 18397 consensus_queue.cc:1035] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 22:33:33.176860 18394 consensus_queue.cc:1035] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:33:33.193276 18400 mvcc.cc:205] Tried to move back new op lower bound from 7315683791542755328 to 7315683791091896320. Current Snapshot: MvccSnapshot[applied={T|T < 7315683791542755328}]
I20260806 22:33:33.241127 17707 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:33:33.241598 17707 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:33:33.247309 17707 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4: Sending DeleteTablet for 3 replicas of tablet 56972b8b8f4c4566a00b4cd811c2b2f1
I20260806 22:33:33.249325 17890 tablet_service.cc:1514] Processing DeleteTablet for tablet 56972b8b8f4c4566a00b4cd811c2b2f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:33 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:52300
I20260806 22:33:33.249418 18072 tablet_service.cc:1514] Processing DeleteTablet for tablet 56972b8b8f4c4566a00b4cd811c2b2f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:33 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:54008
I20260806 22:33:33.250994 18251 tablet_service.cc:1514] Processing DeleteTablet for tablet 56972b8b8f4c4566a00b4cd811c2b2f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:33 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:45830
I20260806 22:33:33.252159 17705 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:33:33.253525 17705 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:33:33.257737 18417 tablet_replica.cc:330] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2: stopping tablet replica
I20260806 22:33:33.258631 18417 raft_consensus.cc:2229] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:33:33.261304 18417 raft_consensus.cc:2258] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:33.269043 18251 tablet_service.cc:1467] Processing CreateTablet for tablet 5de0a79d884643b79d3a6322598d72b3 (DEFAULT_TABLE table=test-table [id=6ed758079e1348518f43f7a1a6626573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:33.269223 17889 tablet_service.cc:1467] Processing CreateTablet for tablet 5de0a79d884643b79d3a6322598d72b3 (DEFAULT_TABLE table=test-table [id=6ed758079e1348518f43f7a1a6626573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:33.270113 18251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5de0a79d884643b79d3a6322598d72b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:33.270272 17889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5de0a79d884643b79d3a6322598d72b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:33.271497 18414 tablet_replica.cc:330] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889: stopping tablet replica
I20260806 22:33:33.272519 18414 raft_consensus.cc:2229] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:33:33.273788 18414 raft_consensus.cc:2258] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:33.268148 18071 tablet_service.cc:1467] Processing CreateTablet for tablet 5de0a79d884643b79d3a6322598d72b3 (DEFAULT_TABLE table=test-table [id=6ed758079e1348518f43f7a1a6626573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:33.275326 18071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5de0a79d884643b79d3a6322598d72b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:33.282749 18414 ts_tablet_manager.cc:1905] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:33:33.285405 18417 ts_tablet_manager.cc:1905] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:33:33.285967 18383 tablet_bootstrap.cc:492] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889: Bootstrap starting.
I20260806 22:33:33.288537 18383 tablet_bootstrap.cc:654] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:33.293116 18382 tablet_bootstrap.cc:492] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2: Bootstrap starting.
I20260806 22:33:33.296118 18382 tablet_bootstrap.cc:654] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:33.298429 18380 tablet_bootstrap.cc:492] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3: Bootstrap starting.
I20260806 22:33:33.298705 18416 tablet_replica.cc:330] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3: stopping tablet replica
I20260806 22:33:33.299466 18416 raft_consensus.cc:2229] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:33:33.302238 18380 tablet_bootstrap.cc:654] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:33.304383 18383 tablet_bootstrap.cc:492] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889: No bootstrap required, opened a new log
I20260806 22:33:33.304735 18383 ts_tablet_manager.cc:1397] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.008s
I20260806 22:33:33.306828 18383 raft_consensus.cc:348] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.307480 18383 raft_consensus.cc:374] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:33.307725 18383 raft_consensus.cc:729] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9559fd469adc4ab090c9c9b63e40e889, State: Initialized, Role: FOLLOWER
I20260806 22:33:33.307899 18417 ts_tablet_manager.cc:1918] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:33:33.308290 18417 log.cc:1196] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/56972b8b8f4c4566a00b4cd811c2b2f1
I20260806 22:33:33.308750 18383 consensus_queue.cc:260] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.309255 18417 ts_tablet_manager.cc:1939] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 8478adc38a7241bd8b78f9b9bd6974d2: Deleting consensus metadata
I20260806 22:33:33.310681 18416 raft_consensus.cc:2258] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:33.312330 17690 catalog_manager.cc:4841] TS 8478adc38a7241bd8b78f9b9bd6974d2 (127.15.198.131:41409): tablet 56972b8b8f4c4566a00b4cd811c2b2f1 (table test-table [id=e67f843f35b546f88dbfd9eadff79d87]) successfully deleted
I20260806 22:33:33.313712 18383 ts_tablet_manager.cc:1428] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260806 22:33:33.316612 18416 ts_tablet_manager.cc:1905] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:33:33.318899 18414 ts_tablet_manager.cc:1918] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:33:33.319249 18414 log.cc:1196] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/56972b8b8f4c4566a00b4cd811c2b2f1
I20260806 22:33:33.320150 18380 tablet_bootstrap.cc:492] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3: No bootstrap required, opened a new log
I20260806 22:33:33.320359 18380 ts_tablet_manager.cc:1397] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.004s
I20260806 22:33:33.320580 18414 ts_tablet_manager.cc:1939] T 56972b8b8f4c4566a00b4cd811c2b2f1 P 9559fd469adc4ab090c9c9b63e40e889: Deleting consensus metadata
I20260806 22:33:33.321650 18380 raft_consensus.cc:348] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.321933 18380 raft_consensus.cc:374] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:33.322073 18380 raft_consensus.cc:729] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a738f94a6da44348a4d1f0cc0b8105d3, State: Initialized, Role: FOLLOWER
I20260806 22:33:33.322615 18380 consensus_queue.cc:260] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.323786 17690 catalog_manager.cc:4841] TS 9559fd469adc4ab090c9c9b63e40e889 (127.15.198.129:44893): tablet 56972b8b8f4c4566a00b4cd811c2b2f1 (table test-table [id=e67f843f35b546f88dbfd9eadff79d87]) successfully deleted
I20260806 22:33:33.324440 18382 tablet_bootstrap.cc:492] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2: No bootstrap required, opened a new log
I20260806 22:33:33.324723 18382 ts_tablet_manager.cc:1397] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.011s
I20260806 22:33:33.326052 18382 raft_consensus.cc:348] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.326516 18382 raft_consensus.cc:374] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:33.326937 18382 raft_consensus.cc:729] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8478adc38a7241bd8b78f9b9bd6974d2, State: Initialized, Role: FOLLOWER
I20260806 22:33:33.327394 18416 ts_tablet_manager.cc:1918] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:33:33.327456 18382 consensus_queue.cc:260] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.327742 18416 log.cc:1196] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/56972b8b8f4c4566a00b4cd811c2b2f1
I20260806 22:33:33.327806 18380 ts_tablet_manager.cc:1428] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260806 22:33:33.328457 18382 ts_tablet_manager.cc:1428] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 22:33:33.329566 18416 ts_tablet_manager.cc:1939] T 56972b8b8f4c4566a00b4cd811c2b2f1 P a738f94a6da44348a4d1f0cc0b8105d3: Deleting consensus metadata
I20260806 22:33:33.332805 17691 catalog_manager.cc:4841] TS a738f94a6da44348a4d1f0cc0b8105d3 (127.15.198.130:43195): tablet 56972b8b8f4c4566a00b4cd811c2b2f1 (table test-table [id=e67f843f35b546f88dbfd9eadff79d87]) successfully deleted
I20260806 22:33:33.401981 18394 raft_consensus.cc:482] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:33:33.402285 18394 raft_consensus.cc:504] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.403439 18394 leader_election.cc:288] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a738f94a6da44348a4d1f0cc0b8105d3 (127.15.198.130:43195), 8478adc38a7241bd8b78f9b9bd6974d2 (127.15.198.131:41409)
I20260806 22:33:33.404441 18284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5de0a79d884643b79d3a6322598d72b3" candidate_uuid: "9559fd469adc4ab090c9c9b63e40e889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" is_pre_election: true
I20260806 22:33:33.404559 18101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5de0a79d884643b79d3a6322598d72b3" candidate_uuid: "9559fd469adc4ab090c9c9b63e40e889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" is_pre_election: true
I20260806 22:33:33.404879 18284 raft_consensus.cc:2454] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9559fd469adc4ab090c9c9b63e40e889 in term 0.
I20260806 22:33:33.404963 18101 raft_consensus.cc:2454] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9559fd469adc4ab090c9c9b63e40e889 in term 0.
I20260806 22:33:33.405632 17817 leader_election.cc:302] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8478adc38a7241bd8b78f9b9bd6974d2, 9559fd469adc4ab090c9c9b63e40e889; no voters:
I20260806 22:33:33.406010 18394 raft_consensus.cc:2781] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:33:33.406299 18394 raft_consensus.cc:482] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:33:33.406513 18394 raft_consensus.cc:3037] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:33.411134 18394 raft_consensus.cc:504] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.412102 18394 leader_election.cc:288] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [CANDIDATE]: Term 1 election: Requested vote from peers a738f94a6da44348a4d1f0cc0b8105d3 (127.15.198.130:43195), 8478adc38a7241bd8b78f9b9bd6974d2 (127.15.198.131:41409)
I20260806 22:33:33.413066 18101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5de0a79d884643b79d3a6322598d72b3" candidate_uuid: "9559fd469adc4ab090c9c9b63e40e889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a738f94a6da44348a4d1f0cc0b8105d3"
I20260806 22:33:33.413097 18284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5de0a79d884643b79d3a6322598d72b3" candidate_uuid: "9559fd469adc4ab090c9c9b63e40e889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8478adc38a7241bd8b78f9b9bd6974d2"
I20260806 22:33:33.413380 18101 raft_consensus.cc:3037] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:33.413431 18284 raft_consensus.cc:3037] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:33.416671 18101 raft_consensus.cc:2454] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9559fd469adc4ab090c9c9b63e40e889 in term 1.
I20260806 22:33:33.417336 17818 leader_election.cc:302] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9559fd469adc4ab090c9c9b63e40e889, a738f94a6da44348a4d1f0cc0b8105d3; no voters:
I20260806 22:33:33.417541 18284 raft_consensus.cc:2454] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9559fd469adc4ab090c9c9b63e40e889 in term 1.
I20260806 22:33:33.417887 18394 raft_consensus.cc:2781] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:33:33.418216 18394 raft_consensus.cc:686] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [term 1 LEADER]: Becoming Leader. State: Replica: 9559fd469adc4ab090c9c9b63e40e889, State: Running, Role: LEADER
I20260806 22:33:33.418661 18394 consensus_queue.cc:237] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.422839 17705 catalog_manager.cc:5495] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9559fd469adc4ab090c9c9b63e40e889 (127.15.198.129). New cstate: current_term: 1 leader_uuid: "9559fd469adc4ab090c9c9b63e40e889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:33:33 dist-test-slave-h3ft krb5kdc[17647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:33.509827 18432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:33.486915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41829 (local address 127.0.0.1:47692)
0806 22:33:33.488882 (+ 1967us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:33.488903 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:33.489103 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:33.489567 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:33.489571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:33.490582 (+ 1011us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:33.490965 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:33.490977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.493054 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.493059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:33.494224 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:33.494238 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.495556 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.495561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:33.495752 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:33.496699 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:33.496719 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:33.499845 (+ 3126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:33.506033 (+ 6188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:33.506054 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:33.506063 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:33.506802 (+ 739us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:33.507378 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:33.507382 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:33.507385 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:33.507618 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:33.508501 (+ 883us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:33.508509 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:33.508763 (+ 254us) client_negotiation.cc:769] Sending connection context
0806 22:33:33.509015 (+ 252us) client_negotiation.cc:240] Negotiation successful
0806 22:33:33.509295 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1823,"thread_start_us":1529,"threads_started":1}
I20260806 22:33:33.510383 18431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:33.486912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47692 (local address 127.15.198.190:41829)
0806 22:33:33.488465 (+ 1553us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:33.488473 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:33.489043 (+ 570us) server_negotiation.cc:407] Connection header received
0806 22:33:33.489253 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:33.489259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:33.489360 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:33.489506 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:33.491259 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.492838 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.494394 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.495344 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.495520 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:33.500218 (+ 4698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:33.500227 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:33.500233 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:33.500280 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:33.505810 (+ 5530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:33.506981 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:33.506989 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:33.506994 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:33.507212 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:33.507769 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:33.507776 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:33.507780 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:33.508298 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:33.508467 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:33.509220 (+ 753us) server_negotiation.cc:299] Negotiation successful
0806 22:33:33.509503 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1241,"thread_start_us":956,"threads_started":1}
I20260806 22:33:33.518823 17705 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:47692:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:33:33.521071 17705 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:33:33.532472 17889 tablet_service.cc:1467] Processing CreateTablet for tablet 9a778e1624074955ba11b20c8f03132f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3860e3754b0d44ecad27abe4dfd7de94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:33:33.532763 18251 tablet_service.cc:1467] Processing CreateTablet for tablet 9a778e1624074955ba11b20c8f03132f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3860e3754b0d44ecad27abe4dfd7de94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:33:33.533471 17889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a778e1624074955ba11b20c8f03132f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:33.533532 18251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a778e1624074955ba11b20c8f03132f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:33.534147 18071 tablet_service.cc:1467] Processing CreateTablet for tablet 9a778e1624074955ba11b20c8f03132f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3860e3754b0d44ecad27abe4dfd7de94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:33:33.534934 18071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a778e1624074955ba11b20c8f03132f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:33.544282 18383 tablet_bootstrap.cc:492] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889: Bootstrap starting.
I20260806 22:33:33.546272 18380 tablet_bootstrap.cc:492] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3: Bootstrap starting.
I20260806 22:33:33.547210 18383 tablet_bootstrap.cc:654] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:33.548679 18382 tablet_bootstrap.cc:492] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2: Bootstrap starting.
I20260806 22:33:33.549007 18380 tablet_bootstrap.cc:654] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:33.553375 18382 tablet_bootstrap.cc:654] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:33.561259 18380 tablet_bootstrap.cc:492] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3: No bootstrap required, opened a new log
I20260806 22:33:33.561318 18382 tablet_bootstrap.cc:492] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2: No bootstrap required, opened a new log
I20260806 22:33:33.561709 18382 ts_tablet_manager.cc:1397] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.002s
I20260806 22:33:33.561736 18380 ts_tablet_manager.cc:1397] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260806 22:33:33.563378 18382 raft_consensus.cc:348] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.563764 18382 raft_consensus.cc:374] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:33.563934 18382 raft_consensus.cc:729] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8478adc38a7241bd8b78f9b9bd6974d2, State: Initialized, Role: FOLLOWER
I20260806 22:33:33.564002 18380 raft_consensus.cc:348] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.564579 18380 raft_consensus.cc:374] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:33.564666 18383 tablet_bootstrap.cc:492] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889: No bootstrap required, opened a new log
I20260806 22:33:33.564425 18382 consensus_queue.cc:260] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.564812 18380 raft_consensus.cc:729] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a738f94a6da44348a4d1f0cc0b8105d3, State: Initialized, Role: FOLLOWER
I20260806 22:33:33.565155 18383 ts_tablet_manager.cc:1397] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.015s
I20260806 22:33:33.565150 18390 tablet_replica.cc:439] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } } }
I20260806 22:33:33.565528 18390 tablet_replica.cc:442] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:33:33.565318 18380 consensus_queue.cc:260] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.566187 18383 raft_consensus.cc:348] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.566174 18391 tablet_replica.cc:439] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } } }
I20260806 22:33:33.566563 18383 raft_consensus.cc:374] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:33.566663 18383 raft_consensus.cc:729] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9559fd469adc4ab090c9c9b63e40e889, State: Initialized, Role: FOLLOWER
I20260806 22:33:33.566762 18391 tablet_replica.cc:442] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:33:33.566962 18383 consensus_queue.cc:260] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.567446 18394 tablet_replica.cc:439] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } } }
I20260806 22:33:33.567799 18394 tablet_replica.cc:442] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:33:33.570516 18380 ts_tablet_manager.cc:1428] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3: Time spent starting tablet: real 0.008s user 0.000s sys 0.003s
I20260806 22:33:33.570815 18382 ts_tablet_manager.cc:1428] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2: Time spent starting tablet: real 0.009s user 0.004s sys 0.000s
I20260806 22:33:33.571390 18383 ts_tablet_manager.cc:1428] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260806 22:33:33.810623 18391 raft_consensus.cc:482] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:33:33.811077 18391 raft_consensus.cc:504] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.813272 18391 leader_election.cc:288] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9559fd469adc4ab090c9c9b63e40e889 (127.15.198.129:44893), 8478adc38a7241bd8b78f9b9bd6974d2 (127.15.198.131:41409)
I20260806 22:33:33.822831 18394 raft_consensus.cc:482] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:33:33.823288 18394 raft_consensus.cc:504] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.824554 18438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:33.813696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:44893 (local address 127.15.198.130:44563)
0806 22:33:33.814487 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:33.814522 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:33.814768 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:33.816143 (+ 1375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:33.816149 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:33.816185 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:33.816889 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:33.816908 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.818887 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.818894 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:33.821051 (+ 2157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:33.821067 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.822961 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.822968 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:33.823201 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:33.823251 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 22:33:33.823806 (+ 555us) client_negotiation.cc:240] Negotiation successful
0806 22:33:33.824016 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":1,"thread_start_us":413,"threads_started":1}
I20260806 22:33:33.825114 18439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:33.813914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:44563 (local address 127.15.198.129:44893)
0806 22:33:33.815607 (+ 1693us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:33.815615 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:33.815641 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:33:33.815830 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:33.815840 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:33.815971 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:33.816101 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:33.817175 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.818693 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.821286 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.822462 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.822722 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:33.822825 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:33.823448 (+ 623us) server_negotiation.cc:299] Negotiation successful
0806 22:33:33.823657 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1470,"thread_start_us":1255,"threads_started":1}
I20260806 22:33:33.825847 18101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a778e1624074955ba11b20c8f03132f" candidate_uuid: "9559fd469adc4ab090c9c9b63e40e889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" is_pre_election: true
I20260806 22:33:33.826253 18101 raft_consensus.cc:2454] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9559fd469adc4ab090c9c9b63e40e889 in term 0.
I20260806 22:33:33.826556 18284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a778e1624074955ba11b20c8f03132f" candidate_uuid: "9559fd469adc4ab090c9c9b63e40e889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" is_pre_election: true
I20260806 22:33:33.826886 17818 leader_election.cc:302] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9559fd469adc4ab090c9c9b63e40e889, a738f94a6da44348a4d1f0cc0b8105d3; no voters:
I20260806 22:33:33.826747 17914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a778e1624074955ba11b20c8f03132f" candidate_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9559fd469adc4ab090c9c9b63e40e889" is_pre_election: true
I20260806 22:33:33.826987 18284 raft_consensus.cc:2454] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9559fd469adc4ab090c9c9b63e40e889 in term 0.
I20260806 22:33:33.827296 17914 raft_consensus.cc:2454] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a738f94a6da44348a4d1f0cc0b8105d3 in term 0.
I20260806 22:33:33.827874 18397 raft_consensus.cc:2781] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:33:33.828058 18397 raft_consensus.cc:482] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:33:33.828213 18397 raft_consensus.cc:3037] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:33.828245 18003 leader_election.cc:302] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9559fd469adc4ab090c9c9b63e40e889, a738f94a6da44348a4d1f0cc0b8105d3; no voters:
I20260806 22:33:33.828269 18441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:33.815296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:52715 (local address 127.15.198.131:41409)
0806 22:33:33.816981 (+ 1685us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:33.816991 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:33.817017 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:33:33.817114 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:33.817122 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:33.817219 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:33.817359 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:33.818191 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.819377 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.822321 (+ 2944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.824059 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.824255 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:33.824338 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:33.826233 (+ 1895us) server_negotiation.cc:299] Negotiation successful
0806 22:33:33.827638 (+ 1405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1489,"thread_start_us":1278,"threads_started":1}
I20260806 22:33:33.829074 18440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:33.814761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:41409 (local address 127.15.198.130:52715)
0806 22:33:33.815803 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:33.815825 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:33.815978 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:33.817379 (+ 1401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:33.817385 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:33.817428 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:33.817998 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:33.818010 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.819569 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.819575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:33.822112 (+ 2537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:33.822132 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.824268 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.824274 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:33.825678 (+ 1404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:33.825720 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 22:33:33.825835 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 22:33:33.826013 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":887,"mutex_wait_us":1,"thread_start_us":714,"threads_started":1}
I20260806 22:33:33.825129 18394 leader_election.cc:288] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a738f94a6da44348a4d1f0cc0b8105d3 (127.15.198.130:43195), 8478adc38a7241bd8b78f9b9bd6974d2 (127.15.198.131:41409)
I20260806 22:33:33.829083 18391 raft_consensus.cc:2781] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:33:33.829725 18391 raft_consensus.cc:482] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:33:33.829871 18391 raft_consensus.cc:3037] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:33.830708 18284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a778e1624074955ba11b20c8f03132f" candidate_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" is_pre_election: true
I20260806 22:33:33.831068 18284 raft_consensus.cc:2454] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a738f94a6da44348a4d1f0cc0b8105d3 in term 0.
I20260806 22:33:33.832863 18397 raft_consensus.cc:504] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.833832 18397 leader_election.cc:288] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [CANDIDATE]: Term 1 election: Requested vote from peers a738f94a6da44348a4d1f0cc0b8105d3 (127.15.198.130:43195), 8478adc38a7241bd8b78f9b9bd6974d2 (127.15.198.131:41409)
I20260806 22:33:33.834398 18101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a778e1624074955ba11b20c8f03132f" candidate_uuid: "9559fd469adc4ab090c9c9b63e40e889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a738f94a6da44348a4d1f0cc0b8105d3"
I20260806 22:33:33.834728 18284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a778e1624074955ba11b20c8f03132f" candidate_uuid: "9559fd469adc4ab090c9c9b63e40e889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8478adc38a7241bd8b78f9b9bd6974d2"
I20260806 22:33:33.835044 18284 raft_consensus.cc:3037] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:33.835289 18391 raft_consensus.cc:504] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.836072 18101 raft_consensus.cc:2379] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9559fd469adc4ab090c9c9b63e40e889 in current term 1: Already voted for candidate a738f94a6da44348a4d1f0cc0b8105d3 in this term.
I20260806 22:33:33.837167 17914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a778e1624074955ba11b20c8f03132f" candidate_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9559fd469adc4ab090c9c9b63e40e889"
I20260806 22:33:33.837612 17914 raft_consensus.cc:2379] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a738f94a6da44348a4d1f0cc0b8105d3 in current term 1: Already voted for candidate 9559fd469adc4ab090c9c9b63e40e889 in this term.
I20260806 22:33:33.838495 18391 leader_election.cc:288] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [CANDIDATE]: Term 1 election: Requested vote from peers 9559fd469adc4ab090c9c9b63e40e889 (127.15.198.129:44893), 8478adc38a7241bd8b78f9b9bd6974d2 (127.15.198.131:41409)
I20260806 22:33:33.839084 18284 raft_consensus.cc:2454] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9559fd469adc4ab090c9c9b63e40e889 in term 1.
I20260806 22:33:33.839687 17817 leader_election.cc:302] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8478adc38a7241bd8b78f9b9bd6974d2, 9559fd469adc4ab090c9c9b63e40e889; no voters: a738f94a6da44348a4d1f0cc0b8105d3
I20260806 22:33:33.839792 18282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a778e1624074955ba11b20c8f03132f" candidate_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8478adc38a7241bd8b78f9b9bd6974d2"
I20260806 22:33:33.840232 18397 raft_consensus.cc:2781] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:33:33.840260 18282 raft_consensus.cc:2379] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a738f94a6da44348a4d1f0cc0b8105d3 in current term 1: Already voted for candidate 9559fd469adc4ab090c9c9b63e40e889 in this term.
I20260806 22:33:33.840435 18397 raft_consensus.cc:686] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [term 1 LEADER]: Becoming Leader. State: Replica: 9559fd469adc4ab090c9c9b63e40e889, State: Running, Role: LEADER
I20260806 22:33:33.840827 18397 consensus_queue.cc:237] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } }
I20260806 22:33:33.841670 18003 leader_election.cc:302] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a738f94a6da44348a4d1f0cc0b8105d3; no voters: 8478adc38a7241bd8b78f9b9bd6974d2, 9559fd469adc4ab090c9c9b63e40e889
I20260806 22:33:33.842309 18391 raft_consensus.cc:2726] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 22:33:33.843619 18394 tablet_replica.cc:439] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889: TxnStatusTablet state changed. Reason: New leader 9559fd469adc4ab090c9c9b63e40e889. Latest consensus state: current_term: 1 leader_uuid: "9559fd469adc4ab090c9c9b63e40e889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } } }
I20260806 22:33:33.844007 18394 tablet_replica.cc:442] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:33:33.845160 18446 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:33:33.845592 18446 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:33:33.845968 17707 catalog_manager.cc:5495] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9559fd469adc4ab090c9c9b63e40e889 (127.15.198.129). New cstate: current_term: 1 leader_uuid: "9559fd469adc4ab090c9c9b63e40e889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } health_report { overall_health: UNKNOWN } } }
I20260806 22:33:33.874768 18394 consensus_queue.cc:1035] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:33:33.882531 18397 consensus_queue.cc:1035] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:33:33.941772 18439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:33.933616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52312 (local address 127.15.198.129:44893)
0806 22:33:33.933933 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:33.933941 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:33.934310 (+ 369us) server_negotiation.cc:407] Connection header received
0806 22:33:33.934700 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:33.934705 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:33.934802 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:33.934895 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:33.935591 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.936714 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.939073 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.939749 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.939971 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:33.940854 (+ 883us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:33.941016 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:33.941111 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:33.941378 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 22:33:33.941488 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 22:33:33.942246 18432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:33.933785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:44893 (local address 127.0.0.1:52312)
0806 22:33:33.934142 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:33.934169 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:33.934431 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:33.934981 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:33.934988 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:33.935033 (+ 45us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:33:33.935392 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:33.935409 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.936895 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.936902 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:33.938778 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:33.938802 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:33.940045 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:33.940054 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:33.940421 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:33.940549 (+ 128us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:33:33.941158 (+ 609us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:33:33.941194 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:33:33.941302 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 22:33:33.941571 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 22:33:33.948861 18101 raft_consensus.cc:1261] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3 [term 1 FOLLOWER]: Refusing update from remote peer 9559fd469adc4ab090c9c9b63e40e889: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:33:33.948992 18282 raft_consensus.cc:1261] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2 [term 1 FOLLOWER]: Refusing update from remote peer 9559fd469adc4ab090c9c9b63e40e889: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:33:33.949878 18397 consensus_queue.cc:1035] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:33:33.950279 18394 consensus_queue.cc:1035] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:33:33.954979 18390 tablet_replica.cc:439] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2: TxnStatusTablet state changed. Reason: New leader 9559fd469adc4ab090c9c9b63e40e889. Latest consensus state: current_term: 1 leader_uuid: "9559fd469adc4ab090c9c9b63e40e889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } } }
I20260806 22:33:33.955444 18390 tablet_replica.cc:442] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:33:33.959481 18390 tablet_replica.cc:439] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9559fd469adc4ab090c9c9b63e40e889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } } }
I20260806 22:33:33.959910 18390 tablet_replica.cc:442] T 9a778e1624074955ba11b20c8f03132f P 8478adc38a7241bd8b78f9b9bd6974d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:33:33.961971 18452 mvcc.cc:205] Tried to move back new op lower bound from 7315683794724904960 to 7315683794295726080. Current Snapshot: MvccSnapshot[applied={T|T < 7315683794724904960 or (T in {7315683794724904960})}]
I20260806 22:33:33.964573 18394 tablet_replica.cc:439] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9559fd469adc4ab090c9c9b63e40e889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } } }
I20260806 22:33:33.965150 18394 tablet_replica.cc:442] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:33:33.968089 18391 tablet_replica.cc:439] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3: TxnStatusTablet state changed. Reason: New leader 9559fd469adc4ab090c9c9b63e40e889. Latest consensus state: current_term: 1 leader_uuid: "9559fd469adc4ab090c9c9b63e40e889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } } }
I20260806 22:33:33.968556 18391 tablet_replica.cc:442] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:33:33.972431 18397 tablet_replica.cc:439] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9559fd469adc4ab090c9c9b63e40e889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } } }
I20260806 22:33:33.973116 18397 tablet_replica.cc:442] T 9a778e1624074955ba11b20c8f03132f P 9559fd469adc4ab090c9c9b63e40e889: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:33:33.978574 18391 tablet_replica.cc:439] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9559fd469adc4ab090c9c9b63e40e889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9559fd469adc4ab090c9c9b63e40e889" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 44893 } } peers { permanent_uuid: "a738f94a6da44348a4d1f0cc0b8105d3" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43195 } } peers { permanent_uuid: "8478adc38a7241bd8b78f9b9bd6974d2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41409 } } }
I20260806 22:33:33.979130 18391 tablet_replica.cc:442] T 9a778e1624074955ba11b20c8f03132f P a738f94a6da44348a4d1f0cc0b8105d3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:33:34.007368 18439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:33.998611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:36063 (local address 127.15.198.129:44893)
0806 22:33:33.998841 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:33.998851 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:33.999934 (+ 1083us) server_negotiation.cc:407] Connection header received
0806 22:33:34.000154 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:34.000160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:34.000441 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:34.000565 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:34.001452 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:34.002739 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:34.004655 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:34.005537 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:34.005687 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:34.006337 (+ 650us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:34.006490 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:34.006612 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:34.006781 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 22:33:34.006864 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 22:33:34.007503 18464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:33.998646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:44893 (local address 127.15.198.129:36063)
0806 22:33:33.999774 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:33.999817 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:33.999995 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:34.000582 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:34.000585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:34.000599 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:33:34.001249 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:34.001260 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:34.002893 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:34.002898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:34.004502 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:34.004517 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:34.005743 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:34.005748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:34.005918 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:34.006017 (+ 99us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:33:34.006609 (+ 592us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:33:34.006621 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 22:33:34.006738 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 22:33:34.006897 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"mutex_wait_us":41,"thread_start_us":654,"threads_started":1}
I20260806 22:33:34.102955 17709 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:33:34.103565 17709 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:33:34.108366 17709 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1c028194e9b4a54b9bcd78c2b567fe4: Sending DeleteTablet for 3 replicas of tablet 5de0a79d884643b79d3a6322598d72b3
I20260806 22:33:34.109782 18251 tablet_service.cc:1514] Processing DeleteTablet for tablet 5de0a79d884643b79d3a6322598d72b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:34 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:45830
I20260806 22:33:34.109896 18071 tablet_service.cc:1514] Processing DeleteTablet for tablet 5de0a79d884643b79d3a6322598d72b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:34 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:54008
I20260806 22:33:34.110877 17889 tablet_service.cc:1514] Processing DeleteTablet for tablet 5de0a79d884643b79d3a6322598d72b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:34 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:52300
I20260806 22:33:34.120914 18469 tablet_replica.cc:330] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2: stopping tablet replica
I20260806 22:33:34.121335 18469 raft_consensus.cc:2229] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:33:34.121853 18469 raft_consensus.cc:2258] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:34.122537 18471 tablet_replica.cc:330] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3: stopping tablet replica
I20260806 22:33:34.122843 18471 raft_consensus.cc:2229] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:33:34.123274 18471 raft_consensus.cc:2258] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:34.124596 18472 tablet_replica.cc:330] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889: stopping tablet replica
I20260806 22:33:34.125048 18472 raft_consensus.cc:2229] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:33:34.125077 18471 ts_tablet_manager.cc:1905] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:33:34.125118 18469 ts_tablet_manager.cc:1905] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:33:34.125720 18472 raft_consensus.cc:2258] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:34.127327 18472 ts_tablet_manager.cc:1905] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:33:34.133694 18471 ts_tablet_manager.cc:1918] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:33:34.133908 18471 log.cc:1196] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/5de0a79d884643b79d3a6322598d72b3
I20260806 22:33:34.134428 18471 ts_tablet_manager.cc:1939] T 5de0a79d884643b79d3a6322598d72b3 P a738f94a6da44348a4d1f0cc0b8105d3: Deleting consensus metadata
I20260806 22:33:34.136003 17691 catalog_manager.cc:4841] TS a738f94a6da44348a4d1f0cc0b8105d3 (127.15.198.130:43195): tablet 5de0a79d884643b79d3a6322598d72b3 (table test-table [id=6ed758079e1348518f43f7a1a6626573]) successfully deleted
I20260806 22:33:34.136703 18469 ts_tablet_manager.cc:1918] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:33:34.136916 18469 log.cc:1196] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/5de0a79d884643b79d3a6322598d72b3
I20260806 22:33:34.137593 18469 ts_tablet_manager.cc:1939] T 5de0a79d884643b79d3a6322598d72b3 P 8478adc38a7241bd8b78f9b9bd6974d2: Deleting consensus metadata
I20260806 22:33:34.139119 17690 catalog_manager.cc:4841] TS 8478adc38a7241bd8b78f9b9bd6974d2 (127.15.198.131:41409): tablet 5de0a79d884643b79d3a6322598d72b3 (table test-table [id=6ed758079e1348518f43f7a1a6626573]) successfully deleted
I20260806 22:33:34.142738 18472 ts_tablet_manager.cc:1918] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:33:34.142969 18472 log.cc:1196] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/5de0a79d884643b79d3a6322598d72b3
I20260806 22:33:34.143639 18472 ts_tablet_manager.cc:1939] T 5de0a79d884643b79d3a6322598d72b3 P 9559fd469adc4ab090c9c9b63e40e889: Deleting consensus metadata
I20260806 22:33:34.145143 17690 catalog_manager.cc:4841] TS 9559fd469adc4ab090c9c9b63e40e889 (127.15.198.129:44893): tablet 5de0a79d884643b79d3a6322598d72b3 (table test-table [id=6ed758079e1348518f43f7a1a6626573]) successfully deleted
Aug 06 22:33:34 dist-test-slave-h3ft krb5kdc[17647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055612, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:33:34.155205 18439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:34.122788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52326 (local address 127.15.198.129:44893)
0806 22:33:34.123084 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:34.123093 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:34.131824 (+ 8731us) server_negotiation.cc:407] Connection header received
0806 22:33:34.132223 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:34.132230 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:34.132347 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:34.132433 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:34.136387 (+ 3954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:34.137676 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:34.139961 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:34.140800 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:34.140961 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:34.147163 (+ 6202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:34.147180 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:34.147202 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:34.147247 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:34.151240 (+ 3993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:34.152201 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:34.152211 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:34.152214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:34.152443 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:34.152981 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:34.152987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:34.152992 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:34.153678 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:34.154078 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:34.154442 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 22:33:34.154726 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
W20260806 22:33:34.156241 17969 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52326
I20260806 22:33:34.185370 18479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:34.160854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47696 (local address 127.15.198.190:41829)
0806 22:33:34.167622 (+ 6768us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:34.167628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:34.167657 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:33:34.167743 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:34.167748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:34.167857 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:34.168080 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:34.170008 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:34.171421 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:34.173185 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:34.173863 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:34.174023 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:34.177572 (+ 3549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:34.177579 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:34.177582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:34.177639 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:34.181447 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:34.182399 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:34.182410 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:34.182412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:34.182600 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:34.183239 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:34.183244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:34.183249 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:34.183632 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:34.184035 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:34.184459 (+ 424us) server_negotiation.cc:299] Negotiation successful
0806 22:33:34.184817 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6612,"thread_start_us":6386,"threads_started":1}
W20260806 22:33:34.186256 17709 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47696
I20260806 22:33:34.188961 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 17798
I20260806 22:33:34.213003 18439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:34.191163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52328 (local address 127.15.198.129:44893)
0806 22:33:34.191415 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:34.191422 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:34.192346 (+ 924us) server_negotiation.cc:407] Connection header received
0806 22:33:34.192593 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:34.192600 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:34.192697 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:34.192811 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:34.194597 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:34.196082 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:34.198203 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:34.199155 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:34.199322 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:34.203300 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:34.203310 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:34.203315 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:34.203389 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:34.208408 (+ 5019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:34.209680 (+ 1272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:34.209699 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:34.209708 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:34.209959 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:34.210486 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:34.210492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:34.210498 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:34.211120 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:34.211504 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:34.211863 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 22:33:34.212251 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260806 22:33:34.213907 17969 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52328
W20260806 22:33:34.214665 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 17798: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:33:34.214963 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 17798
I20260806 22:33:34.264201 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 17980
Aug 06 22:33:34 dist-test-slave-h3ft krb5kdc[17647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055612, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.198.130@KRBTEST.COM
I20260806 22:33:34.284475 18481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:34.265181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54016 (local address 127.15.198.130:43195)
0806 22:33:34.265799 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:34.265804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:34.265823 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:33:34.265893 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:34.265898 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:34.265978 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:34.266081 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:34.267835 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:34.268741 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:34.270355 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:34.270959 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:34.271271 (+ 312us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:34.276179 (+ 4908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:34.276200 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:34.276224 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:34.276274 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:34.280475 (+ 4201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:34.281586 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:34.281596 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:34.281598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:34.281789 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:34.282366 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:34.282371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:34.282373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:34.282947 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:34.283340 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:34.283743 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 22:33:34.284062 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":338,"threads_started":1}
W20260806 22:33:34.285523 18151 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54016
W20260806 22:33:34.286494 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 17980: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:33:34.286751 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 17980
I20260806 22:33:34.314620 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 18165
Aug 06 22:33:34 dist-test-slave-h3ft krb5kdc[17647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055612, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.198.131@KRBTEST.COM
I20260806 22:33:34.334566 18441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:34.315886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45840 (local address 127.15.198.131:41409)
0806 22:33:34.316185 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:34.316192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:34.316220 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:33:34.316316 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:34.316328 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:34.316423 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:34.316575 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:34.318150 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:34.318942 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:34.320507 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:34.320999 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:34.321158 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:34.326865 (+ 5707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:34.326892 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:34.326918 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:34.326975 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:34.330883 (+ 3908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:34.331913 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:34.331925 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:34.331929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:34.332104 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:34.332556 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:34.332562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:34.332583 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:34.333062 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:34.333503 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:34.333828 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 22:33:34.334151 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260806 22:33:34.335632 18345 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45840
W20260806 22:33:34.336609 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 18165: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:33:34.336812 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 18165
I20260806 22:33:34.369138 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 17665
I20260806 22:33:34.370774 17761 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:34.519744 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 17665
2026-08-06T22:33:34Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (6670 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:33:34 dist-test-slave-h3ft krb5kdc[18485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:33:34 dist-test-slave-h3ft krb5kdc[18485](info): set up 2 sockets
Aug 06 22:33:34 dist-test-slave-h3ft krb5kdc[18485](info): commencing operation
krb5kdc: starting...
W20260806 22:33:35.902110 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.317s user 0.002s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:33:36 dist-test-slave-h3ft krb5kdc[18485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:33:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:33:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:33:36.200337 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:35053
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:38899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:35053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:36.644232 18504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:36.644804 18504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:36.645008 18504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:36.654438 18504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:33:36.654723 18504 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:33:36.654847 18504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:36.655025 18504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:36.655135 18504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:33:36.655249 18504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:33:36.667624 18504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:38899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:35053
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:35053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:36.669683 18504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:36.671729 18504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:36.687810 18510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:36.688102 18514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:36.687927 18512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:36.689626 18504 server_base.cc:1034] running on GCE node
I20260806 22:33:36.690953 18504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:36.693244 18504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:36.694674 18504 hybrid_clock.cc:648] HybridClock initialized: now 1786055616694558 us; error 82 us; skew 500 ppm
I20260806 22:33:36.695295 18504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:36 dist-test-slave-h3ft krb5kdc[18485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055616, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:36.700018 18504 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:33:36.702190 18504 webserver.cc:460] Webserver started at http://127.15.198.190:39207/ using document root <none> and password file <none>
I20260806 22:33:36.703480 18504 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:36.703645 18504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:36.704082 18504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:36.708775 18504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "119ee579be3e4781ae9b556391f6a48f"
format_stamp: "Formatted at 2026-08-06 22:33:36 on dist-test-slave-h3ft"
server_key: "7bd50e5a043570f90ba378ec6e1fe411"
server_key_iv: "f33a249142cdc91c0a24f61084f83758"
server_key_version: "encryptionkey@0"
I20260806 22:33:36.709885 18504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "119ee579be3e4781ae9b556391f6a48f"
format_stamp: "Formatted at 2026-08-06 22:33:36 on dist-test-slave-h3ft"
server_key: "7bd50e5a043570f90ba378ec6e1fe411"
server_key_iv: "f33a249142cdc91c0a24f61084f83758"
server_key_version: "encryptionkey@0"
I20260806 22:33:36.718209 18504 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260806 22:33:36.723758 18523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:36.726135 18504 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:33:36.726482 18504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "119ee579be3e4781ae9b556391f6a48f"
format_stamp: "Formatted at 2026-08-06 22:33:36 on dist-test-slave-h3ft"
server_key: "7bd50e5a043570f90ba378ec6e1fe411"
server_key_iv: "f33a249142cdc91c0a24f61084f83758"
server_key_version: "encryptionkey@0"
I20260806 22:33:36.726866 18504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:36.763500 18504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:36.770838 18504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:36.771286 18504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:36.800745 18504 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:35053
I20260806 22:33:36.800765 18609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:35053 every 8 connection(s)
I20260806 22:33:36.802736 18504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:33:36.804914 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 18504
I20260806 22:33:36.805202 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:33:36.806318 16154 external_mini_cluster.cc:1428] Setting key 51ff24702e1f5ad3218952c64435ce3b
I20260806 22:33:36.810384 18611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 22:33:36 dist-test-slave-h3ft krb5kdc[18485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:36.827374 18611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f: Bootstrap starting.
I20260806 22:33:36.833436 18611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:36.835443 18611 log.cc:824] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:33:36.840842 18614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:36.809527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49186 (local address 127.15.198.190:35053)
0806 22:33:36.810849 (+ 1322us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:36.810866 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:36.810933 (+ 67us) server_negotiation.cc:407] Connection header received
0806 22:33:36.813147 (+ 2214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:36.813187 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:36.813738 (+ 551us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:36.814250 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:36.816937 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:36.818899 (+ 1962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:36.820751 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:36.821750 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:36.822051 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:36.827779 (+ 5728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:36.827792 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:36.827816 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:36.827879 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:36.833059 (+ 5180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:36.834467 (+ 1408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:36.834480 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:36.834530 (+ 50us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:36.834863 (+ 333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:36.835598 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:36.835607 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:36.835614 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:36.836721 (+ 1107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:36.837422 (+ 701us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:36.838213 (+ 791us) server_negotiation.cc:299] Negotiation successful
0806 22:33:36.838871 (+ 658us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1021,"thread_start_us":678,"threads_started":1}
I20260806 22:33:36.840994 18611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f: No bootstrap required, opened a new log
I20260806 22:33:36.854568 18611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "119ee579be3e4781ae9b556391f6a48f" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 35053 } }
I20260806 22:33:36.855319 18611 raft_consensus.cc:374] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:36.855434 18611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 119ee579be3e4781ae9b556391f6a48f, State: Initialized, Role: FOLLOWER
I20260806 22:33:36.856344 18611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "119ee579be3e4781ae9b556391f6a48f" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 35053 } }
I20260806 22:33:36.856665 18611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:33:36.856798 18611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:33:36.857129 18611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:36.860863 18611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "119ee579be3e4781ae9b556391f6a48f" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 35053 } }
I20260806 22:33:36.861737 18611 leader_election.cc:302] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 119ee579be3e4781ae9b556391f6a48f; no voters:
I20260806 22:33:36.862586 18611 leader_election.cc:288] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:33:36.862833 18616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:33:36.863809 18616 raft_consensus.cc:686] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [term 1 LEADER]: Becoming Leader. State: Replica: 119ee579be3e4781ae9b556391f6a48f, State: Running, Role: LEADER
I20260806 22:33:36.864634 18616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "119ee579be3e4781ae9b556391f6a48f" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 35053 } }
I20260806 22:33:36.865799 18611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:33:36.869167 18622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 119ee579be3e4781ae9b556391f6a48f. Latest consensus state: current_term: 1 leader_uuid: "119ee579be3e4781ae9b556391f6a48f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "119ee579be3e4781ae9b556391f6a48f" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 35053 } } }
I20260806 22:33:36.869858 18622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [sys.catalog]: This master's current role is: LEADER
I20260806 22:33:36.870654 18619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "119ee579be3e4781ae9b556391f6a48f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "119ee579be3e4781ae9b556391f6a48f" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 35053 } } }
I20260806 22:33:36.872008 18619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f [sys.catalog]: This master's current role is: LEADER
I20260806 22:33:36.872985 18631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:33:36.879328 18631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:33:36.892560 18631 catalog_manager.cc:1347] Generated new cluster ID: b15bcfa2411a4544b8d18e9bbe620562
I20260806 22:33:36.892853 18631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:33:36.909458 18631 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:33:36.911716 18631 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:33:36.924638 18631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f: Generated new TSK 0
I20260806 22:33:36.926455 18631 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
I20260806 22:33:37.012984 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:35053
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:38899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:37.484405 18649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:37.484822 18649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:37.485067 18649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:37.493862 18649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:37.494052 18649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:37.494242 18649 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:33:37.494314 18649 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:33:37.494443 18649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:33:37.507647 18649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:38899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.198.190:35053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:37.509667 18649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:37.511660 18649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:37.532846 18657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:37.533401 18663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:37.533684 18654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:37.535856 18649 server_base.cc:1034] running on GCE node
I20260806 22:33:37.536895 18649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:37.538262 18649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:37.539722 18649 hybrid_clock.cc:648] HybridClock initialized: now 1786055617539653 us; error 42 us; skew 500 ppm
I20260806 22:33:37.540256 18649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:37 dist-test-slave-h3ft krb5kdc[18485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055617, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:37.545252 18649 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:33:37.547760 18649 webserver.cc:460] Webserver started at http://127.15.198.129:34831/ using document root <none> and password file <none>
I20260806 22:33:37.548852 18649 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:37.549211 18649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:37.549850 18649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:37.554276 18649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c"
format_stamp: "Formatted at 2026-08-06 22:33:37 on dist-test-slave-h3ft"
server_key: "949cbf29b3f7ca6294bb090584e7398f"
server_key_iv: "e6946ad2a0b9dc23805478633e97bcb6"
server_key_version: "encryptionkey@0"
I20260806 22:33:37.555469 18649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c"
format_stamp: "Formatted at 2026-08-06 22:33:37 on dist-test-slave-h3ft"
server_key: "949cbf29b3f7ca6294bb090584e7398f"
server_key_iv: "e6946ad2a0b9dc23805478633e97bcb6"
server_key_version: "encryptionkey@0"
I20260806 22:33:37.564021 18649 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260806 22:33:37.570020 18671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:37.572329 18649 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 22:33:37.572775 18649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c"
format_stamp: "Formatted at 2026-08-06 22:33:37 on dist-test-slave-h3ft"
server_key: "949cbf29b3f7ca6294bb090584e7398f"
server_key_iv: "e6946ad2a0b9dc23805478633e97bcb6"
server_key_version: "encryptionkey@0"
I20260806 22:33:37.573156 18649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:37.609215 18649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:37.615392 18649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:37.615808 18649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:37.619518 18649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:37.619704 18649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:37.619870 18649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:37.619989 18649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:33:37 dist-test-slave-h3ft krb5kdc[18485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055617, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:37.652333 18680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:37.620381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:43523 (local address 127.15.198.190:35053)
0806 22:33:37.621127 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:37.621135 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:37.622626 (+ 1491us) server_negotiation.cc:407] Connection header received
0806 22:33:37.625158 (+ 2532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:37.625164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:37.625286 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:37.625437 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:37.628492 (+ 3055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:37.629549 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:37.633556 (+ 4007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:37.634483 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:37.634662 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:37.640824 (+ 6162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:37.640837 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:37.640847 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:37.640926 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:37.645880 (+ 4954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:37.647513 (+ 1633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:37.647523 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:37.647528 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:37.647769 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:37.648499 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:37.648510 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:37.648516 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:37.649154 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:37.649391 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:37.651312 (+ 1921us) server_negotiation.cc:299] Negotiation successful
0806 22:33:37.651751 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":416,"threads_started":1}
I20260806 22:33:37.655154 18681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:37.621243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:35053 (local address 127.15.198.129:43523)
0806 22:33:37.622346 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:37.622408 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:37.624734 (+ 2326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:37.625813 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:37.625832 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:37.626912 (+ 1080us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:37.628215 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:37.628257 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:37.629811 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:37.629818 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:37.633257 (+ 3439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:37.633283 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:37.634976 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:37.634992 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:37.635325 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:37.636554 (+ 1229us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:37.636619 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:37.640350 (+ 3731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:37.646232 (+ 5882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:37.646252 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:37.646283 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:37.647291 (+ 1008us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:37.647900 (+ 609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:37.647906 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:37.647911 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:37.648291 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:37.649499 (+ 1208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:37.649522 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:37.650467 (+ 945us) client_negotiation.cc:769] Sending connection context
0806 22:33:37.651037 (+ 570us) client_negotiation.cc:240] Negotiation successful
0806 22:33:37.652056 (+ 1019us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"thread_start_us":474,"threads_started":1}
I20260806 22:33:37.685453 18649 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:37659
I20260806 22:33:37.685453 18831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:37659 every 8 connection(s)
I20260806 22:33:37.687769 18649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:33:37.690192 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:37.666450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:35053 (local address 127.15.198.129:39039)
0806 22:33:37.667544 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:37.667576 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:37.667837 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:37.668340 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:37.668345 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:37.669134 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:37.670193 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:37.670219 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:37.671838 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:37.671846 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:37.674813 (+ 2967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:37.674867 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:37.676287 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:37.676298 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:37.676633 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:37.677855 (+ 1222us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:37.677955 (+ 100us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:37.680226 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:37.685999 (+ 5773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:37.686005 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:37.686008 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:37.686584 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:37.687278 (+ 694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:37.687282 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:37.687285 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:37.687468 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:37.688383 (+ 915us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:37.688387 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:37.688883 (+ 496us) client_negotiation.cc:769] Sending connection context
0806 22:33:37.689105 (+ 222us) client_negotiation.cc:240] Negotiation successful
0806 22:33:37.689611 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"thread_start_us":433,"threads_started":1}
I20260806 22:33:37.690893 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 18649
I20260806 22:33:37.691244 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:33:37.691267 18832 heartbeater.cc:347] Connected to a master server at 127.15.198.190:35053
I20260806 22:33:37.692142 16154 external_mini_cluster.cc:1428] Setting key beb6950399dde048be91232faecd13a5
I20260806 22:33:37.692379 18680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:37.665935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:39039 (local address 127.15.198.190:35053)
0806 22:33:37.666261 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:37.666268 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:37.667736 (+ 1468us) server_negotiation.cc:407] Connection header received
0806 22:33:37.668035 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:37.668042 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:37.668151 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:37.668286 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:37.670500 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:37.671553 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:37.675223 (+ 3670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:37.675921 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:37.676149 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:37.680603 (+ 4454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:37.680614 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:37.680620 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:37.680690 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:37.685734 (+ 5044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:37.686806 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:37.686814 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:37.686818 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:37.687095 (+ 277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:37.687646 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:37.687655 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:37.687661 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:37.688195 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:37.688406 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:37.689330 (+ 924us) server_negotiation.cc:299] Negotiation successful
0806 22:33:37.689793 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 22:33:37.691716 18832 heartbeater.cc:464] Registering TS with master...
I20260806 22:33:37.694607 18832 heartbeater.cc:511] Master 127.15.198.190:35053 requested a full tablet report, sending...
I20260806 22:33:37.698009 18546 ts_manager.cc:194] Registered new tserver with Master: 2e73a6bb8c9d475aab217e3b5d81bd5c (127.15.198.129:37659)
I20260806 22:33:37.700304 18546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:43523
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
I20260806 22:33:37.764863 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:35053
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:38899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:38.245294 18837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:38.245775 18837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:38.246006 18837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:38.255633 18837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:38.255831 18837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:38.256045 18837 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:33:38.256167 18837 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:33:38.256309 18837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:33:38.268800 18837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:38899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.198.190:35053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:38.270933 18837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:38.272864 18837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:38.291911 18844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:38.291875 18845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:38.291880 18847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:38.299365 18837 server_base.cc:1034] running on GCE node
I20260806 22:33:38.300467 18837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:38.301901 18837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:38.303363 18837 hybrid_clock.cc:648] HybridClock initialized: now 1786055618303244 us; error 108 us; skew 500 ppm
I20260806 22:33:38.303838 18837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:38 dist-test-slave-h3ft krb5kdc[18485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055618, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:38.308773 18837 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:33:38.311340 18837 webserver.cc:460] Webserver started at http://127.15.198.130:39351/ using document root <none> and password file <none>
I20260806 22:33:38.312435 18837 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:38.312650 18837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:38.313156 18837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:38.318087 18837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "6cfe51d728f54515b0f5af9732db8b91"
format_stamp: "Formatted at 2026-08-06 22:33:38 on dist-test-slave-h3ft"
server_key: "1093a5ef8c8a31daba77515ba9a8973a"
server_key_iv: "3e41bc39f11d65177b1ea4d56e4fc2b5"
server_key_version: "encryptionkey@0"
I20260806 22:33:38.319339 18837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "6cfe51d728f54515b0f5af9732db8b91"
format_stamp: "Formatted at 2026-08-06 22:33:38 on dist-test-slave-h3ft"
server_key: "1093a5ef8c8a31daba77515ba9a8973a"
server_key_iv: "3e41bc39f11d65177b1ea4d56e4fc2b5"
server_key_version: "encryptionkey@0"
I20260806 22:33:38.326742 18837 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260806 22:33:38.331679 18857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:38.334306 18837 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260806 22:33:38.334604 18837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "6cfe51d728f54515b0f5af9732db8b91"
format_stamp: "Formatted at 2026-08-06 22:33:38 on dist-test-slave-h3ft"
server_key: "1093a5ef8c8a31daba77515ba9a8973a"
server_key_iv: "3e41bc39f11d65177b1ea4d56e4fc2b5"
server_key_version: "encryptionkey@0"
I20260806 22:33:38.334933 18837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:38.387948 18837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:38.394286 18837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:38.394749 18837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:38.398576 18837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:38.398772 18837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:38.398969 18837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:38.399140 18837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 06 22:33:38 dist-test-slave-h3ft krb5kdc[18485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055618, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:38.435194 18870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:38.400562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:35053 (local address 127.15.198.130:49551)
0806 22:33:38.401800 (+ 1238us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:38.401875 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:38.405819 (+ 3944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:38.407236 (+ 1417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:38.407248 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:38.408433 (+ 1185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:38.409612 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:38.409667 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:38.412018 (+ 2351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:38.412023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:38.414505 (+ 2482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:38.414526 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:38.415909 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:38.415925 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:38.416504 (+ 579us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:38.417855 (+ 1351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:38.417912 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:38.422834 (+ 4922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:38.428451 (+ 5617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:38.428474 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:38.428511 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:38.429450 (+ 939us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:38.430118 (+ 668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:38.430126 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:38.430131 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:38.430690 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:38.431691 (+ 1001us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:38.431706 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:38.432412 (+ 706us) client_negotiation.cc:769] Sending connection context
0806 22:33:38.432847 (+ 435us) client_negotiation.cc:240] Negotiation successful
0806 22:33:38.433525 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"thread_start_us":455,"threads_started":1}
I20260806 22:33:38.436251 18869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:38.399658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:49551 (local address 127.15.198.190:35053)
0806 22:33:38.400467 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:38.400475 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:38.402076 (+ 1601us) server_negotiation.cc:407] Connection header received
0806 22:33:38.406418 (+ 4342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:38.406429 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:38.406585 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:38.406774 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:38.410030 (+ 3256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:38.411761 (+ 1731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:38.414871 (+ 3110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:38.415499 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:38.415687 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:38.423337 (+ 7650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:38.423346 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:38.423350 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:38.423407 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:38.428190 (+ 4783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:38.429684 (+ 1494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:38.429691 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:38.429694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:38.429911 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:38.430899 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:38.430907 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:38.430915 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:38.431343 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:38.431562 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:38.435318 (+ 3756us) server_negotiation.cc:299] Negotiation successful
0806 22:33:38.435728 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":475,"threads_started":1}
I20260806 22:33:38.460821 18837 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:34223
I20260806 22:33:38.460898 19025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:34223 every 8 connection(s)
I20260806 22:33:38.463335 18837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:33:38.464035 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 18837
I20260806 22:33:38.464277 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:33:38.465066 16154 external_mini_cluster.cc:1428] Setting key 3ab98fc5a6a01bf0905d7b718382bd10
I20260806 22:33:38.466899 19026 heartbeater.cc:347] Connected to a master server at 127.15.198.190:35053
I20260806 22:33:38.467327 19026 heartbeater.cc:464] Registering TS with master...
I20260806 22:33:38.468559 18869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:38.444722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:40981 (local address 127.15.198.190:35053)
0806 22:33:38.445026 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:38.445037 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:38.445806 (+ 769us) server_negotiation.cc:407] Connection header received
0806 22:33:38.446135 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:38.446140 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:38.446237 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:38.446349 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:38.448472 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:38.449418 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:38.452375 (+ 2957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:38.453392 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:38.453599 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:38.458211 (+ 4612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:38.458223 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:38.458228 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:38.458288 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:38.462949 (+ 4661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:38.464338 (+ 1389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:38.464350 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:38.464357 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:38.464610 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:38.465280 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:38.465309 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:38.465316 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:38.466152 (+ 836us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:38.466377 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:38.467484 (+ 1107us) server_negotiation.cc:299] Negotiation successful
0806 22:33:38.467974 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 22:33:38.468612 19026 heartbeater.cc:511] Master 127.15.198.190:35053 requested a full tablet report, sending...
I20260806 22:33:38.470621 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:38.444702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:35053 (local address 127.15.198.130:40981)
0806 22:33:38.445605 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:38.445638 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:38.445913 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:38.446463 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:38.446471 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:38.447481 (+ 1010us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:38.448186 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:38.448218 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:38.449631 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:38.449637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:38.452054 (+ 2417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:38.452080 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:38.454139 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:38.454159 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:38.454451 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:38.455708 (+ 1257us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:38.455744 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:38.457744 (+ 2000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:38.463243 (+ 5499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:38.463249 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:38.463253 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:38.464085 (+ 832us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:38.464838 (+ 753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:38.464847 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:38.464854 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:38.465099 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:38.466517 (+ 1418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:38.466526 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:38.467034 (+ 508us) client_negotiation.cc:769] Sending connection context
0806 22:33:38.468914 (+ 1880us) client_negotiation.cc:240] Negotiation successful
0806 22:33:38.469661 (+ 747us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":437,"threads_started":1}
I20260806 22:33:38.472075 18547 ts_manager.cc:194] Registered new tserver with Master: 6cfe51d728f54515b0f5af9732db8b91 (127.15.198.130:34223)
I20260806 22:33:38.473752 18547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:49551
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
I20260806 22:33:38.536581 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:35053
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:38899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:33:38.705418 18832 heartbeater.cc:503] Master 127.15.198.190:35053 was elected leader, sending a full tablet report...
W20260806 22:33:39.019228 19033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:39.019780 19033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:39.019922 19033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:39.028805 19033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:39.028998 19033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:39.029110 19033 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:33:39.029170 19033 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:33:39.029250 19033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:33:39.043895 19033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:38899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.198.190:35053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:39.045943 19033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:39.048256 19033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:39.067718 19039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:39.067950 19038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:39.067775 19041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:39.069900 19033 server_base.cc:1034] running on GCE node
I20260806 22:33:39.070935 19033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:39.072716 19033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:39.074144 19033 hybrid_clock.cc:648] HybridClock initialized: now 1786055619074084 us; error 58 us; skew 500 ppm
I20260806 22:33:39.074615 19033 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:39 dist-test-slave-h3ft krb5kdc[18485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055619, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:39.079666 19033 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:33:39.082129 19033 webserver.cc:460] Webserver started at http://127.15.198.131:43829/ using document root <none> and password file <none>
I20260806 22:33:39.083256 19033 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:39.083415 19033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:39.083855 19033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:39.088212 19033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "c59ad9b6f91a4009901f247db3efc4ac"
format_stamp: "Formatted at 2026-08-06 22:33:39 on dist-test-slave-h3ft"
server_key: "032434ade01457994faa40a8b5a57637"
server_key_iv: "ea3d18ade0ead813858ddf128478f2f3"
server_key_version: "encryptionkey@0"
I20260806 22:33:39.089386 19033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "c59ad9b6f91a4009901f247db3efc4ac"
format_stamp: "Formatted at 2026-08-06 22:33:39 on dist-test-slave-h3ft"
server_key: "032434ade01457994faa40a8b5a57637"
server_key_iv: "ea3d18ade0ead813858ddf128478f2f3"
server_key_version: "encryptionkey@0"
I20260806 22:33:39.097177 19033 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.004s
I20260806 22:33:39.103245 19050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:39.105626 19033 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260806 22:33:39.105985 19033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "c59ad9b6f91a4009901f247db3efc4ac"
format_stamp: "Formatted at 2026-08-06 22:33:39 on dist-test-slave-h3ft"
server_key: "032434ade01457994faa40a8b5a57637"
server_key_iv: "ea3d18ade0ead813858ddf128478f2f3"
server_key_version: "encryptionkey@0"
I20260806 22:33:39.106292 19033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:39.125948 19033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:39.133057 19033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:39.133468 19033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:39.137413 19033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:39.137544 19033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:39.137705 19033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:39.137804 19033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:33:39 dist-test-slave-h3ft krb5kdc[18485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055619, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:39.172886 19066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.140306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:35789 (local address 127.15.198.190:35053)
0806 22:33:39.141009 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:39.141018 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:39.141259 (+ 241us) server_negotiation.cc:407] Connection header received
0806 22:33:39.145682 (+ 4423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:39.145693 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:39.145837 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:39.145996 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:39.149239 (+ 3243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.150534 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.153401 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.154094 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.154261 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.161246 (+ 6985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:39.161254 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:39.161259 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:39.161335 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:39.166453 (+ 5118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:39.168643 (+ 2190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:39.168655 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:39.168663 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:39.168982 (+ 319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:39.169714 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:39.169724 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:39.169731 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:39.170437 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:39.170655 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:39.171878 (+ 1223us) server_negotiation.cc:299] Negotiation successful
0806 22:33:39.172249 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":1598,"threads_started":1}
I20260806 22:33:39.174669 19064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.139565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:35053 (local address 127.15.198.131:35789)
0806 22:33:39.140997 (+ 1432us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:39.141071 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:39.145152 (+ 4081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:39.146514 (+ 1362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:39.146536 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:39.147661 (+ 1125us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:39.148951 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.148981 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.150833 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.150841 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.153140 (+ 2299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.153169 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.154432 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.154445 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.154738 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.155759 (+ 1021us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:39.155817 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:39.160756 (+ 4939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:39.166736 (+ 5980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:39.166760 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:39.166800 (+ 40us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:39.168302 (+ 1502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:39.169156 (+ 854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:39.169161 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:39.169164 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:39.169534 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:39.170619 (+ 1085us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:39.170632 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:39.171324 (+ 692us) client_negotiation.cc:769] Sending connection context
0806 22:33:39.171755 (+ 431us) client_negotiation.cc:240] Negotiation successful
0806 22:33:39.172602 (+ 847us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"spinlock_wait_cycles":33664,"thread_start_us":802,"threads_started":1}
I20260806 22:33:39.204617 19033 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:44935
I20260806 22:33:39.204706 19222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:44935 every 8 connection(s)
I20260806 22:33:39.206524 19033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:33:39.210412 19223 heartbeater.cc:347] Connected to a master server at 127.15.198.190:35053
I20260806 22:33:39.210779 19223 heartbeater.cc:464] Registering TS with master...
I20260806 22:33:39.212028 19066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.187678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:60389 (local address 127.15.198.190:35053)
0806 22:33:39.188029 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:39.188039 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:39.189175 (+ 1136us) server_negotiation.cc:407] Connection header received
0806 22:33:39.189507 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:39.189515 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:39.189687 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:39.189857 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:39.192396 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.193313 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.197258 (+ 3945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.198107 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.198340 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.202432 (+ 4092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:39.202440 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:39.202445 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:39.202500 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:39.206794 (+ 4294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:39.208193 (+ 1399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:39.208202 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:39.208206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:39.208640 (+ 434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:39.209941 (+ 1301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:39.209978 (+ 37us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:39.209983 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:39.210546 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:39.210736 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:39.211299 (+ 563us) server_negotiation.cc:299] Negotiation successful
0806 22:33:39.211604 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260806 22:33:39.212225 19223 heartbeater.cc:511] Master 127.15.198.190:35053 requested a full tablet report, sending...
I20260806 22:33:39.212248 19176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.187672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:35053 (local address 127.15.198.131:60389)
0806 22:33:39.188978 (+ 1306us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:39.189007 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:39.189315 (+ 308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:39.189948 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:39.189958 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:39.191240 (+ 1282us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:39.192107 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.192131 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.193639 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.193646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.196816 (+ 3170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.196857 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.198777 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.198790 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.199133 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.200388 (+ 1255us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:39.200437 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:39.202049 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:39.207039 (+ 4990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:39.207048 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:39.207054 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:39.207926 (+ 872us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:39.209012 (+ 1086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:39.209023 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:39.209034 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:39.209582 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:39.210681 (+ 1099us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:39.210686 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:39.210944 (+ 258us) client_negotiation.cc:769] Sending connection context
0806 22:33:39.211172 (+ 228us) client_negotiation.cc:240] Negotiation successful
0806 22:33:39.211662 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1047,"thread_start_us":790,"threads_started":1}
I20260806 22:33:39.214661 18546 ts_manager.cc:194] Registered new tserver with Master: c59ad9b6f91a4009901f247db3efc4ac (127.15.198.131:44935)
I20260806 22:33:39.214797 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 19033
I20260806 22:33:39.215035 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:33:39.215601 16154 external_mini_cluster.cc:1428] Setting key 290e1e87ca3e7db365806a829f8f5c1d
I20260806 22:33:39.216192 18546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:35789
I20260806 22:33:39.219955 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:33:39 dist-test-slave-h3ft krb5kdc[18485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:33:39 dist-test-slave-h3ft krb5kdc[18485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:39.301734 19066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.281999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43666 (local address 127.15.198.190:35053)
0806 22:33:39.282371 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:39.282380 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:39.282860 (+ 480us) server_negotiation.cc:407] Connection header received
0806 22:33:39.283281 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:39.283288 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:39.283385 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:39.283531 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:39.285534 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.286536 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.288300 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.288806 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.288973 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.294327 (+ 5354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:39.294335 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:39.294339 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:39.294404 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:39.298010 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:39.298994 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:39.299052 (+ 58us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:39.299057 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:39.299265 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:39.299782 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:39.299787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:39.299789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:39.300194 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:39.300549 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:39.300965 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 22:33:39.301267 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260806 22:33:39.308233 18546 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:33:39.317106 18546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:33:39.364495 19240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.337426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:37659 (local address 127.0.0.1:55050)
0806 22:33:39.352793 (+ 15367us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:39.352830 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:39.353092 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:39.355624 (+ 2532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:39.355630 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:39.355657 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:39.356036 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.356046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.358217 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.358222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.361448 (+ 3226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.361472 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.363170 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.363180 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.363464 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.363553 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 22:33:39.363689 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 22:33:39.363922 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15070,"mutex_wait_us":73,"thread_start_us":15826,"threads_started":1}
I20260806 22:33:39.364985 19238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.338419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:34223 (local address 127.0.0.1:51430)
0806 22:33:39.354601 (+ 16182us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:39.354625 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:39.354761 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:39.356892 (+ 2131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:39.356897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:39.356925 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:39.357326 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.357337 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.359816 (+ 2479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.359823 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.361508 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.361528 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.364078 (+ 2550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.364085 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.364261 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.364298 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 22:33:39.364398 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 22:33:39.364557 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16021,"mutex_wait_us":7,"thread_start_us":9581,"threads_started":1}
I20260806 22:33:39.365463 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.338772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55050 (local address 127.15.198.129:37659)
0806 22:33:39.345738 (+ 6966us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:39.345756 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:39.353035 (+ 7279us) server_negotiation.cc:407] Connection header received
0806 22:33:39.354210 (+ 1175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:39.354229 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:39.354676 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:39.355293 (+ 617us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:39.356229 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.358028 (+ 1799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.361649 (+ 3621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.362853 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.364522 (+ 1669us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.364755 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:39.364866 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 22:33:39.365101 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6742,"thread_start_us":6548,"threads_started":1}
I20260806 22:33:39.366145 19246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.339289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51430 (local address 127.15.198.130:34223)
0806 22:33:39.355671 (+ 16382us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:39.355687 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:39.355737 (+ 50us) server_negotiation.cc:407] Connection header received
0806 22:33:39.355909 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:39.355921 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:39.356288 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:39.356587 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:39.357552 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.359644 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.361761 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.363438 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.363631 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.363879 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:39.365571 (+ 1692us) server_negotiation.cc:299] Negotiation successful
0806 22:33:39.365786 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16070,"thread_start_us":16904,"threads_started":1}
I20260806 22:33:39.366743 19244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.337220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:44935 (local address 127.0.0.1:49052)
0806 22:33:39.348761 (+ 11541us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:39.348845 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:39.349218 (+ 373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:39.358427 (+ 9209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:39.358434 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:39.358468 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:39.359084 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.359100 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.361456 (+ 2356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.361462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.363742 (+ 2280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.363759 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.365923 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.365930 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.366112 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.366146 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 22:33:39.366218 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:33:39.366411 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11060,"thread_start_us":17670,"threads_started":1}
I20260806 22:33:39.367617 19242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.337600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49052 (local address 127.15.198.131:44935)
0806 22:33:39.357205 (+ 19605us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:39.357215 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:39.357255 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:33:39.357375 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:39.357385 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:39.357671 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:39.358019 (+ 348us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:39.359375 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.361250 (+ 1875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.363970 (+ 2720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.365719 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.365929 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.366182 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:39.366344 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 22:33:39.366570 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19401,"thread_start_us":19108,"threads_started":1}
I20260806 22:33:39.370857 18735 tablet_service.cc:1467] Processing CreateTablet for tablet 106201d85c5344b5bd80e4e3a27c831e (DEFAULT_TABLE table=test-table [id=fe4b26b80e824ff997e608880d76fddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:39.370867 18921 tablet_service.cc:1467] Processing CreateTablet for tablet 106201d85c5344b5bd80e4e3a27c831e (DEFAULT_TABLE table=test-table [id=fe4b26b80e824ff997e608880d76fddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:39.372650 18921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 106201d85c5344b5bd80e4e3a27c831e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:39.372650 18735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 106201d85c5344b5bd80e4e3a27c831e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:39.378794 19122 tablet_service.cc:1467] Processing CreateTablet for tablet 106201d85c5344b5bd80e4e3a27c831e (DEFAULT_TABLE table=test-table [id=fe4b26b80e824ff997e608880d76fddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:39.381130 19122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 106201d85c5344b5bd80e4e3a27c831e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:39.389698 19248 tablet_bootstrap.cc:492] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c: Bootstrap starting.
I20260806 22:33:39.392696 19249 tablet_bootstrap.cc:492] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91: Bootstrap starting.
I20260806 22:33:39.395439 19248 tablet_bootstrap.cc:654] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:39.397168 19248 log.cc:824] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:33:39.397518 19249 tablet_bootstrap.cc:654] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:39.399207 19249 log.cc:824] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91: Log is configured to *not* fsync() on all Append() calls
I20260806 22:33:39.400379 19255 tablet_bootstrap.cc:492] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac: Bootstrap starting.
I20260806 22:33:39.402243 19248 tablet_bootstrap.cc:492] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c: No bootstrap required, opened a new log
I20260806 22:33:39.402676 19248 ts_tablet_manager.cc:1397] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260806 22:33:39.405725 19255 tablet_bootstrap.cc:654] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:39.406641 19249 tablet_bootstrap.cc:492] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91: No bootstrap required, opened a new log
I20260806 22:33:39.407094 19249 ts_tablet_manager.cc:1397] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.000s
I20260806 22:33:39.407553 19255 log.cc:824] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac: Log is configured to *not* fsync() on all Append() calls
I20260806 22:33:39.411237 19255 tablet_bootstrap.cc:492] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac: No bootstrap required, opened a new log
I20260806 22:33:39.411602 19255 ts_tablet_manager.cc:1397] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.008s
I20260806 22:33:39.415166 19248 raft_consensus.cc:348] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } }
I20260806 22:33:39.415834 19248 raft_consensus.cc:374] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:39.415989 19248 raft_consensus.cc:729] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e73a6bb8c9d475aab217e3b5d81bd5c, State: Initialized, Role: FOLLOWER
I20260806 22:33:39.416837 19248 consensus_queue.cc:260] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } }
I20260806 22:33:39.419335 19248 ts_tablet_manager.cc:1428] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c: Time spent starting tablet: real 0.017s user 0.015s sys 0.005s
I20260806 22:33:39.419281 19249 raft_consensus.cc:348] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } }
I20260806 22:33:39.419960 19249 raft_consensus.cc:374] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:39.420137 19249 raft_consensus.cc:729] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cfe51d728f54515b0f5af9732db8b91, State: Initialized, Role: FOLLOWER
I20260806 22:33:39.421087 19249 consensus_queue.cc:260] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } }
I20260806 22:33:39.421522 19255 raft_consensus.cc:348] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } }
I20260806 22:33:39.422228 19255 raft_consensus.cc:374] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:39.422412 19255 raft_consensus.cc:729] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59ad9b6f91a4009901f247db3efc4ac, State: Initialized, Role: FOLLOWER
I20260806 22:33:39.422541 19026 heartbeater.cc:503] Master 127.15.198.190:35053 was elected leader, sending a full tablet report...
I20260806 22:33:39.423208 19255 consensus_queue.cc:260] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } }
I20260806 22:33:39.424083 19249 ts_tablet_manager.cc:1428] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91: Time spent starting tablet: real 0.017s user 0.012s sys 0.003s
I20260806 22:33:39.425339 19223 heartbeater.cc:503] Master 127.15.198.190:35053 was elected leader, sending a full tablet report...
I20260806 22:33:39.426105 19255 ts_tablet_manager.cc:1428] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac: Time spent starting tablet: real 0.014s user 0.005s sys 0.008s
W20260806 22:33:39.444911 18833 tablet.cc:2369] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:33:39.459626 19224 tablet.cc:2369] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:33:39.481462 19028 tablet.cc:2369] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:33:39.503808 19260 raft_consensus.cc:482] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:33:39.504309 19260 raft_consensus.cc:504] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } }
I20260806 22:33:39.506716 19260 leader_election.cc:288] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c59ad9b6f91a4009901f247db3efc4ac (127.15.198.131:44935), 2e73a6bb8c9d475aab217e3b5d81bd5c (127.15.198.129:37659)
I20260806 22:33:39.515988 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.507733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:33527 (local address 127.15.198.129:37659)
0806 22:33:39.508072 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:39.508081 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:39.508679 (+ 598us) server_negotiation.cc:407] Connection header received
0806 22:33:39.509000 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:39.509006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:39.509111 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:39.509234 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:39.509910 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.510748 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.513911 (+ 3163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.514813 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.515056 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.515166 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:39.515522 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 22:33:39.515679 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260806 22:33:39.516192 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.507291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:37659 (local address 127.15.198.130:33527)
0806 22:33:39.508486 (+ 1195us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:39.508509 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:39.508800 (+ 291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:39.509256 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:39.509260 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:39.509286 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:39.509728 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.509740 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.510931 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.510937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.513704 (+ 2767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.513719 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.515120 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.515127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.515315 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.515347 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 22:33:39.515492 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 22:33:39.515658 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":985,"mutex_wait_us":54,"thread_start_us":702,"threads_started":1}
I20260806 22:33:39.516196 19242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.507446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:48541 (local address 127.15.198.131:44935)
0806 22:33:39.507775 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:39.507786 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:39.508920 (+ 1134us) server_negotiation.cc:407] Connection header received
0806 22:33:39.509156 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:39.509161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:39.509238 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:39.509352 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:39.510116 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.510897 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.513895 (+ 2998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.514789 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.515079 (+ 290us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.515184 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:39.515723 (+ 539us) server_negotiation.cc:299] Negotiation successful
0806 22:33:39.515880 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 22:33:39.516314 19263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.507323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:44935 (local address 127.15.198.130:48541)
0806 22:33:39.508767 (+ 1444us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:39.508783 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:39.508970 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:39.509371 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:39.509375 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:39.509422 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:39.509904 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.509914 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.511092 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.511100 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.513696 (+ 2596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.513715 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.515180 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.515186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.515516 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.515546 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 22:33:39.515684 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 22:33:39.515835 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1311,"mutex_wait_us":160,"thread_start_us":973,"threads_started":1}
I20260806 22:33:39.517671 19155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106201d85c5344b5bd80e4e3a27c831e" candidate_uuid: "6cfe51d728f54515b0f5af9732db8b91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad9b6f91a4009901f247db3efc4ac" is_pre_election: true
I20260806 22:33:39.517673 18763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106201d85c5344b5bd80e4e3a27c831e" candidate_uuid: "6cfe51d728f54515b0f5af9732db8b91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" is_pre_election: true
I20260806 22:33:39.518380 18763 raft_consensus.cc:2454] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cfe51d728f54515b0f5af9732db8b91 in term 0.
I20260806 22:33:39.518380 19155 raft_consensus.cc:2454] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cfe51d728f54515b0f5af9732db8b91 in term 0.
I20260806 22:33:39.519275 18861 leader_election.cc:302] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cfe51d728f54515b0f5af9732db8b91, c59ad9b6f91a4009901f247db3efc4ac; no voters:
I20260806 22:33:39.519824 19260 raft_consensus.cc:2781] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:33:39.520018 19260 raft_consensus.cc:482] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:33:39.520143 19260 raft_consensus.cc:3037] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:39.523101 19260 raft_consensus.cc:504] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } }
I20260806 22:33:39.523968 19260 leader_election.cc:288] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [CANDIDATE]: Term 1 election: Requested vote from peers c59ad9b6f91a4009901f247db3efc4ac (127.15.198.131:44935), 2e73a6bb8c9d475aab217e3b5d81bd5c (127.15.198.129:37659)
I20260806 22:33:39.524608 19155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106201d85c5344b5bd80e4e3a27c831e" candidate_uuid: "6cfe51d728f54515b0f5af9732db8b91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad9b6f91a4009901f247db3efc4ac"
I20260806 22:33:39.524665 18763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106201d85c5344b5bd80e4e3a27c831e" candidate_uuid: "6cfe51d728f54515b0f5af9732db8b91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c"
I20260806 22:33:39.524923 18763 raft_consensus.cc:3037] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:39.524947 19155 raft_consensus.cc:3037] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:39.527837 18763 raft_consensus.cc:2454] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cfe51d728f54515b0f5af9732db8b91 in term 1.
I20260806 22:33:39.528142 19155 raft_consensus.cc:2454] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cfe51d728f54515b0f5af9732db8b91 in term 1.
I20260806 22:33:39.528647 18859 leader_election.cc:302] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e73a6bb8c9d475aab217e3b5d81bd5c, 6cfe51d728f54515b0f5af9732db8b91; no voters:
I20260806 22:33:39.529130 19260 raft_consensus.cc:2781] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:33:39.529749 19260 raft_consensus.cc:686] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [term 1 LEADER]: Becoming Leader. State: Replica: 6cfe51d728f54515b0f5af9732db8b91, State: Running, Role: LEADER
I20260806 22:33:39.530535 19260 consensus_queue.cc:237] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } }
I20260806 22:33:39.537324 18545 catalog_manager.cc:5495] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cfe51d728f54515b0f5af9732db8b91 (127.15.198.130). New cstate: current_term: 1 leader_uuid: "6cfe51d728f54515b0f5af9732db8b91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } health_report { overall_health: HEALTHY } } }
Aug 06 22:33:39 dist-test-slave-h3ft krb5kdc[18485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055616, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:39.612274 19275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.594682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:35053 (local address 127.0.0.1:43680)
0806 22:33:39.595558 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:39.595577 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:39.595798 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:39.596187 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:39.596191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:39.597124 (+ 933us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:39.597477 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.597490 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.598647 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.598655 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.600067 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.600081 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.600929 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.600935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.601105 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.602014 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:39.602042 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:39.605083 (+ 3041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:39.608883 (+ 3800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:39.608906 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:39.608914 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:39.609586 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:39.610107 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:39.610111 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:39.610114 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:39.610286 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:39.610977 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:39.610986 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:39.611243 (+ 257us) client_negotiation.cc:769] Sending connection context
0806 22:33:39.611395 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 22:33:39.611828 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":554,"threads_started":1}
I20260806 22:33:39.612290 19066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.594603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43680 (local address 127.15.198.190:35053)
0806 22:33:39.594916 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:39.594921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:39.595692 (+ 771us) server_negotiation.cc:407] Connection header received
0806 22:33:39.595979 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:39.595983 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:39.596047 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:39.596157 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:39.597641 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.598512 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.600225 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.600707 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.600867 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.605357 (+ 4490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:39.605364 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:39.605368 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:39.605412 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:39.608686 (+ 3274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:39.609763 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:39.609768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:39.609771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:39.609963 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:39.610437 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:39.610441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:39.610443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:39.610822 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:39.610958 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:39.611530 (+ 572us) server_negotiation.cc:299] Negotiation successful
0806 22:33:39.612042 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":217,"server-negotiator.queue_time_us":71}
I20260806 22:33:39.619467 18545 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:43680:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:33:39.621022 18545 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:33:39.631285 19122 tablet_service.cc:1467] Processing CreateTablet for tablet 5b8e6b41c16a49a494f051d679f700b0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=29f0c112330e4c7a845ce5cae3e25808]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:33:39.630865 18735 tablet_service.cc:1467] Processing CreateTablet for tablet 5b8e6b41c16a49a494f051d679f700b0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=29f0c112330e4c7a845ce5cae3e25808]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:33:39.631044 18921 tablet_service.cc:1467] Processing CreateTablet for tablet 5b8e6b41c16a49a494f051d679f700b0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=29f0c112330e4c7a845ce5cae3e25808]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:33:39.632462 18921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b8e6b41c16a49a494f051d679f700b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:39.632463 18735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b8e6b41c16a49a494f051d679f700b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:39.632512 19122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b8e6b41c16a49a494f051d679f700b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:39.641552 19249 tablet_bootstrap.cc:492] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91: Bootstrap starting.
I20260806 22:33:39.641544 19248 tablet_bootstrap.cc:492] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Bootstrap starting.
I20260806 22:33:39.641547 19255 tablet_bootstrap.cc:492] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac: Bootstrap starting.
I20260806 22:33:39.643661 19255 tablet_bootstrap.cc:654] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:39.644034 19249 tablet_bootstrap.cc:654] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:39.644034 19248 tablet_bootstrap.cc:654] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:39.647125 19255 tablet_bootstrap.cc:492] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac: No bootstrap required, opened a new log
I20260806 22:33:39.647897 19255 ts_tablet_manager.cc:1397] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260806 22:33:39.649149 19249 tablet_bootstrap.cc:492] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91: No bootstrap required, opened a new log
I20260806 22:33:39.649375 19249 ts_tablet_manager.cc:1397] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 22:33:39.649111 19255 raft_consensus.cc:348] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:39.649520 19255 raft_consensus.cc:374] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:39.649582 19248 tablet_bootstrap.cc:492] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c: No bootstrap required, opened a new log
I20260806 22:33:39.649658 19255 raft_consensus.cc:729] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59ad9b6f91a4009901f247db3efc4ac, State: Initialized, Role: FOLLOWER
I20260806 22:33:39.649767 19248 ts_tablet_manager.cc:1397] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260806 22:33:39.649972 19255 consensus_queue.cc:260] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:39.650440 19249 raft_consensus.cc:348] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:39.650723 19255 ts_tablet_manager.cc:1428] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260806 22:33:39.650803 19249 raft_consensus.cc:374] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:39.650964 19249 raft_consensus.cc:729] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cfe51d728f54515b0f5af9732db8b91, State: Initialized, Role: FOLLOWER
I20260806 22:33:39.650803 19261 tablet_replica.cc:439] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } }
I20260806 22:33:39.651319 19261 tablet_replica.cc:442] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:33:39.651223 19248 raft_consensus.cc:348] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:39.651360 19249 consensus_queue.cc:260] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:39.651528 19248 raft_consensus.cc:374] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:39.651641 19248 raft_consensus.cc:729] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e73a6bb8c9d475aab217e3b5d81bd5c, State: Initialized, Role: FOLLOWER
I20260806 22:33:39.651830 19260 tablet_replica.cc:439] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } }
I20260806 22:33:39.651885 19248 consensus_queue.cc:260] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:39.652154 19260 tablet_replica.cc:442] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:33:39.652448 19259 tablet_replica.cc:439] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } }
I20260806 22:33:39.653151 19259 tablet_replica.cc:442] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:33:39.653810 19248 ts_tablet_manager.cc:1428] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260806 22:33:39.654729 19249 ts_tablet_manager.cc:1428] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260806 22:33:39.798717 19259 raft_consensus.cc:482] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:33:39.798978 19261 raft_consensus.cc:482] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:33:39.799496 19261 raft_consensus.cc:504] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:39.799523 19259 raft_consensus.cc:504] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:39.803983 19259 leader_election.cc:288] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cfe51d728f54515b0f5af9732db8b91 (127.15.198.130:34223), c59ad9b6f91a4009901f247db3efc4ac (127.15.198.131:44935)
I20260806 22:33:39.804498 19261 leader_election.cc:288] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e73a6bb8c9d475aab217e3b5d81bd5c (127.15.198.129:37659), 6cfe51d728f54515b0f5af9732db8b91 (127.15.198.130:34223)
I20260806 22:33:39.813690 19246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.803594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:49239 (local address 127.15.198.130:34223)
0806 22:33:39.803824 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:39.803830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:39.803858 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:33:39.803959 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:39.803966 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:39.804059 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:39.804182 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:39.805148 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.806519 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.809739 (+ 3220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.811298 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.811494 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.811592 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:39.813100 (+ 1508us) server_negotiation.cc:299] Negotiation successful
0806 22:33:39.813298 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 22:33:39.815112 19284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.802227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:34223 (local address 127.15.198.129:49239)
0806 22:33:39.803536 (+ 1309us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:39.803561 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:39.803776 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:39.804414 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:39.804421 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:39.804457 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:39.804905 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.804920 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.806771 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.806778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.809461 (+ 2683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.809491 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.811858 (+ 2367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.811868 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.812088 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.812129 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:33:39.814327 (+ 2198us) client_negotiation.cc:240] Negotiation successful
0806 22:33:39.814556 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1139,"thread_start_us":1451,"threads_started":1}
I20260806 22:33:39.816816 18950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b8e6b41c16a49a494f051d679f700b0" candidate_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfe51d728f54515b0f5af9732db8b91" is_pre_election: true
I20260806 22:33:39.817672 18950 raft_consensus.cc:2454] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in term 0.
I20260806 22:33:39.819048 18672 leader_election.cc:302] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e73a6bb8c9d475aab217e3b5d81bd5c, 6cfe51d728f54515b0f5af9732db8b91; no voters:
I20260806 22:33:39.820039 19259 raft_consensus.cc:2781] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:33:39.820298 19259 raft_consensus.cc:482] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:33:39.820513 19259 raft_consensus.cc:3037] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:39.824306 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.803093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:37589 (local address 127.15.198.129:37659)
0806 22:33:39.803352 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:39.803360 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:39.814235 (+ 10875us) server_negotiation.cc:407] Connection header received
0806 22:33:39.815689 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:39.815699 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:39.815793 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:39.815861 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:39.816596 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.817969 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.821033 (+ 3064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.822759 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.822938 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.823065 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:39.823740 (+ 675us) server_negotiation.cc:299] Negotiation successful
0806 22:33:39.823923 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 22:33:39.825306 19283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.802226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:37659 (local address 127.15.198.131:37589)
0806 22:33:39.814085 (+ 11859us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:39.814119 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:39.814540 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:39.816081 (+ 1541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:39.816086 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:39.816118 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:39.816391 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.816403 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.818200 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.818207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.820761 (+ 2554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.820785 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.823269 (+ 2484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.823275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.823513 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.823556 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:33:39.824625 (+ 1069us) client_negotiation.cc:240] Negotiation successful
0806 22:33:39.824846 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11673,"thread_start_us":11520,"threads_started":1}
I20260806 22:33:39.825356 19259 raft_consensus.cc:504] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:39.826464 18763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b8e6b41c16a49a494f051d679f700b0" candidate_uuid: "c59ad9b6f91a4009901f247db3efc4ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" is_pre_election: true
I20260806 22:33:39.827212 18763 raft_consensus.cc:2379] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c59ad9b6f91a4009901f247db3efc4ac in current term 1: Already voted for candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in this term.
I20260806 22:33:39.828198 18950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b8e6b41c16a49a494f051d679f700b0" candidate_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfe51d728f54515b0f5af9732db8b91"
I20260806 22:33:39.828537 18950 raft_consensus.cc:3037] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:39.831650 19288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.805640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:49687 (local address 127.15.198.130:34223)
0806 22:33:39.812955 (+ 7315us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:39.812961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:39.821357 (+ 8396us) server_negotiation.cc:407] Connection header received
0806 22:33:39.821956 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:39.821963 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:39.822096 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:39.822188 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:39.823415 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.824906 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.827862 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.829630 (+ 1768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.829856 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.829973 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:39.830832 (+ 859us) server_negotiation.cc:299] Negotiation successful
0806 22:33:39.831094 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7112,"thread_start_us":6763,"threads_started":1}
I20260806 22:33:39.832875 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.804786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:34223 (local address 127.15.198.131:49687)
0806 22:33:39.821208 (+ 16422us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:39.821236 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:39.821757 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:39.822589 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:39.822597 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:39.822645 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:39.823132 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.823146 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.825135 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.825144 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.827618 (+ 2474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.827644 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.830229 (+ 2585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.830237 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.830579 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.830623 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:33:39.832016 (+ 1393us) client_negotiation.cc:240] Negotiation successful
0806 22:33:39.832223 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16254,"thread_start_us":16596,"threads_started":1}
I20260806 22:33:39.833946 19242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.814212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:59601 (local address 127.15.198.131:44935)
0806 22:33:39.814401 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:39.814408 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:39.815252 (+ 844us) server_negotiation.cc:407] Connection header received
0806 22:33:39.815618 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:39.815624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:39.815734 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:39.815813 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:33:39.826625 (+ 10812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.827982 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.830819 (+ 2837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.832396 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.832556 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.832669 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:39.833411 (+ 742us) server_negotiation.cc:299] Negotiation successful
0806 22:33:39.833597 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":60}
I20260806 22:33:39.834034 18949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b8e6b41c16a49a494f051d679f700b0" candidate_uuid: "c59ad9b6f91a4009901f247db3efc4ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfe51d728f54515b0f5af9732db8b91" is_pre_election: true
I20260806 22:33:39.835116 19286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.803402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:44935 (local address 127.15.198.129:59601)
0806 22:33:39.815091 (+ 11689us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:39.815126 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:39.815418 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:39.825733 (+ 10315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:39.825740 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:39.825800 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:33:39.826288 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.826303 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.828192 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.828199 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.830588 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.830609 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.832932 (+ 2323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.832940 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.833179 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.833222 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:33:39.833963 (+ 741us) client_negotiation.cc:240] Negotiation successful
0806 22:33:39.834540 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11399,"thread_start_us":11143,"threads_started":1}
I20260806 22:33:39.835747 19259 leader_election.cc:288] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [CANDIDATE]: Term 1 election: Requested vote from peers 6cfe51d728f54515b0f5af9732db8b91 (127.15.198.130:34223), c59ad9b6f91a4009901f247db3efc4ac (127.15.198.131:44935)
I20260806 22:33:39.836009 18950 raft_consensus.cc:2454] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in term 1.
I20260806 22:33:39.836462 19052 leader_election.cc:302] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c59ad9b6f91a4009901f247db3efc4ac; no voters: 2e73a6bb8c9d475aab217e3b5d81bd5c, 6cfe51d728f54515b0f5af9732db8b91
I20260806 22:33:39.836658 18672 leader_election.cc:302] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e73a6bb8c9d475aab217e3b5d81bd5c, 6cfe51d728f54515b0f5af9732db8b91; no voters:
I20260806 22:33:39.836764 19155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b8e6b41c16a49a494f051d679f700b0" candidate_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad9b6f91a4009901f247db3efc4ac" is_pre_election: true
I20260806 22:33:39.837082 19259 raft_consensus.cc:2781] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:33:39.837107 19155 raft_consensus.cc:2454] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in term 0.
I20260806 22:33:39.837146 19151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b8e6b41c16a49a494f051d679f700b0" candidate_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad9b6f91a4009901f247db3efc4ac"
I20260806 22:33:39.837438 19261 raft_consensus.cc:3037] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:39.838268 19259 raft_consensus.cc:686] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 LEADER]: Becoming Leader. State: Replica: 2e73a6bb8c9d475aab217e3b5d81bd5c, State: Running, Role: LEADER
I20260806 22:33:39.839138 19259 consensus_queue.cc:237] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:39.840806 19261 raft_consensus.cc:2726] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 22:33:39.844326 19151 raft_consensus.cc:2454] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in term 1.
I20260806 22:33:39.844791 19289 tablet_replica.cc:439] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c: TxnStatusTablet state changed. Reason: New leader 2e73a6bb8c9d475aab217e3b5d81bd5c. Latest consensus state: current_term: 1 leader_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } }
I20260806 22:33:39.845572 19289 tablet_replica.cc:442] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:33:39.846984 19293 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:33:39.847414 19293 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:33:39.847275 18546 catalog_manager.cc:5495] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e73a6bb8c9d475aab217e3b5d81bd5c (127.15.198.129). New cstate: current_term: 1 leader_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } health_report { overall_health: UNKNOWN } } }
I20260806 22:33:39.903466 19275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.896087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:37659 (local address 127.0.0.1:55058)
0806 22:33:39.896464 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:39.896506 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:39.896769 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:39.897531 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:39.897537 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:39.897553 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:33:39.897808 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.897819 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.899113 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.899119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.900272 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:39.900290 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.901186 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.901193 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:39.901388 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.901483 (+ 95us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:33:39.902680 (+ 1197us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:33:39.902716 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:33:39.902835 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:33:39.903117 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 22:33:39.903451 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.896458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55058 (local address 127.15.198.129:37659)
0806 22:33:39.896737 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:39.896744 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:39.896771 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:33:39.897147 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:39.897152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:39.897273 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:39.897504 (+ 231us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:39.898001 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.898908 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.900471 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.900965 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.901138 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.901729 (+ 591us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:39.902545 (+ 816us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:39.902733 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:39.902880 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 22:33:39.903030 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 22:33:39.917238 19151 raft_consensus.cc:1261] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Refusing update from remote peer 2e73a6bb8c9d475aab217e3b5d81bd5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:33:39.918706 19289 consensus_queue.cc:1035] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:33:39.919740 18950 raft_consensus.cc:1261] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91 [term 1 FOLLOWER]: Refusing update from remote peer 2e73a6bb8c9d475aab217e3b5d81bd5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:33:39.920935 19289 consensus_queue.cc:1035] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:33:39.939755 19260 tablet_replica.cc:439] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91: TxnStatusTablet state changed. Reason: New leader 2e73a6bb8c9d475aab217e3b5d81bd5c. Latest consensus state: current_term: 1 leader_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } }
I20260806 22:33:39.940248 19260 tablet_replica.cc:442] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:33:39.941282 19261 tablet_replica.cc:439] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac: TxnStatusTablet state changed. Reason: New leader 2e73a6bb8c9d475aab217e3b5d81bd5c. Latest consensus state: current_term: 1 leader_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } }
I20260806 22:33:39.941697 19261 tablet_replica.cc:442] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:33:39.944342 19296 mvcc.cc:205] Tried to move back new op lower bound from 7315683819166261248 to 7315683818866728960. Current Snapshot: MvccSnapshot[applied={T|T < 7315683819166261248}]
I20260806 22:33:39.945043 19260 tablet_replica.cc:439] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } }
I20260806 22:33:39.945425 19260 tablet_replica.cc:442] T 5b8e6b41c16a49a494f051d679f700b0 P 6cfe51d728f54515b0f5af9732db8b91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:33:39.953539 19297 mvcc.cc:205] Tried to move back new op lower bound from 7315683819166261248 to 7315683818866728960. Current Snapshot: MvccSnapshot[applied={T|T < 7315683819166261248}]
I20260806 22:33:39.954326 19298 mvcc.cc:205] Tried to move back new op lower bound from 7315683819166261248 to 7315683818866728960. Current Snapshot: MvccSnapshot[applied={T|T < 7315683819166261248}]
I20260806 22:33:39.957898 19259 tablet_replica.cc:439] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } }
I20260806 22:33:39.958463 19259 tablet_replica.cc:442] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:33:39.960255 19259 tablet_replica.cc:439] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } }
I20260806 22:33:39.960673 19259 tablet_replica.cc:442] T 5b8e6b41c16a49a494f051d679f700b0 P 2e73a6bb8c9d475aab217e3b5d81bd5c: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:33:39.968099 19261 tablet_replica.cc:439] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } } }
I20260806 22:33:39.968673 19261 tablet_replica.cc:442] T 5b8e6b41c16a49a494f051d679f700b0 P c59ad9b6f91a4009901f247db3efc4ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:33:39.991456 19288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:39.978942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51434 (local address 127.15.198.130:34223)
0806 22:33:39.979302 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:39.979313 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:39.980746 (+ 1433us) server_negotiation.cc:407] Connection header received
0806 22:33:39.981082 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:39.981091 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:39.981283 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:39.981529 (+ 246us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:39.982159 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.983581 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.987233 (+ 3652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:39.988092 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:39.988301 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:39.989012 (+ 711us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:39.990192 (+ 1180us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:39.990401 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:39.990588 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 22:33:39.990736 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260806 22:33:40.024857 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:40.014607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:37659 (local address 127.15.198.130:44593)
0806 22:33:40.015759 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:40.015785 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:40.016019 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:40.016564 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:40.016572 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:40.016589 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:33:40.017300 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:40.017316 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:40.019653 (+ 2337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:40.019660 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:40.021459 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:40.021481 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:40.022970 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:40.022978 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:40.023237 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:40.023368 (+ 131us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:33:40.024057 (+ 689us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:33:40.024072 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 22:33:40.024184 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 22:33:40.024411 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"thread_start_us":618,"threads_started":1}
I20260806 22:33:40.024966 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:40.014595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:44593 (local address 127.15.198.129:37659)
0806 22:33:40.014874 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:40.014884 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:40.015938 (+ 1054us) server_negotiation.cc:407] Connection header received
0806 22:33:40.016232 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:40.016245 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:40.016395 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:40.016520 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:40.017532 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:40.019462 (+ 1930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:40.021681 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:40.022730 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:40.022926 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:40.023683 (+ 757us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:40.023919 (+ 236us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:40.024041 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:40.024241 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 22:33:40.024386 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 22:33:40.042630 19260 consensus_queue.cc:1035] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:33:40.046849 19155 raft_consensus.cc:1261] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Refusing update from remote peer 6cfe51d728f54515b0f5af9732db8b91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:33:40.047673 19260 consensus_queue.cc:1035] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:33:40.146167 19288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:40.132360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:58485 (local address 127.15.198.130:34223)
0806 22:33:40.132697 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:40.132707 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:40.136243 (+ 3536us) server_negotiation.cc:407] Connection header received
0806 22:33:40.136513 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:40.136522 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:40.136691 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:40.136818 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:40.137999 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:40.139555 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:40.142560 (+ 3005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:40.143694 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:40.143838 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:40.144868 (+ 1030us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:40.145104 (+ 236us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:40.145193 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:40.145571 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 22:33:40.145711 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260806 22:33:40.146409 19327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:40.132983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:34223 (local address 127.15.198.129:58485)
0806 22:33:40.136054 (+ 3071us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:40.136082 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:40.136310 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:40.137096 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:40.137104 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:40.137124 (+ 20us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:33:40.137725 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:40.137742 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:40.139851 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:40.139859 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:40.142272 (+ 2413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:40.142299 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:40.144141 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:40.144150 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:40.144425 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:40.144568 (+ 143us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:33:40.145307 (+ 739us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:33:40.145323 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 22:33:40.145488 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 22:33:40.145768 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2733,"thread_start_us":2529,"threads_started":1}
I20260806 22:33:40.209794 18545 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:33:40.210366 18545 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:33:40.215953 18545 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f: Sending DeleteTablet for 3 replicas of tablet 106201d85c5344b5bd80e4e3a27c831e
I20260806 22:33:40.217491 18735 tablet_service.cc:1514] Processing DeleteTablet for tablet 106201d85c5344b5bd80e4e3a27c831e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:40 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:55050
I20260806 22:33:40.217479 19122 tablet_service.cc:1514] Processing DeleteTablet for tablet 106201d85c5344b5bd80e4e3a27c831e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:40 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:49052
I20260806 22:33:40.222175 18921 tablet_service.cc:1514] Processing DeleteTablet for tablet 106201d85c5344b5bd80e4e3a27c831e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:40 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:51430
I20260806 22:33:40.230298 18546 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:33:40.231823 18546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:33:40.246958 19121 tablet_service.cc:1467] Processing CreateTablet for tablet f742ba066eb047e08ec26dd07f4d1aeb (DEFAULT_TABLE table=test-table [id=29174abda4544625b1d904306a858bee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:40.247267 18920 tablet_service.cc:1467] Processing CreateTablet for tablet f742ba066eb047e08ec26dd07f4d1aeb (DEFAULT_TABLE table=test-table [id=29174abda4544625b1d904306a858bee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:40.247891 19121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f742ba066eb047e08ec26dd07f4d1aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:40.247960 18920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f742ba066eb047e08ec26dd07f4d1aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:40.252985 18733 tablet_service.cc:1467] Processing CreateTablet for tablet f742ba066eb047e08ec26dd07f4d1aeb (DEFAULT_TABLE table=test-table [id=29174abda4544625b1d904306a858bee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:40.253767 18733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f742ba066eb047e08ec26dd07f4d1aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:40.272209 19338 tablet_bootstrap.cc:492] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac: Bootstrap starting.
I20260806 22:33:40.280776 19338 tablet_bootstrap.cc:654] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:40.299219 19338 tablet_bootstrap.cc:492] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac: No bootstrap required, opened a new log
I20260806 22:33:40.299727 19338 ts_tablet_manager.cc:1397] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac: Time spent bootstrapping tablet: real 0.028s user 0.006s sys 0.005s
I20260806 22:33:40.301218 19338 raft_consensus.cc:348] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:40.301801 19338 raft_consensus.cc:374] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:40.301997 19338 raft_consensus.cc:729] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59ad9b6f91a4009901f247db3efc4ac, State: Initialized, Role: FOLLOWER
I20260806 22:33:40.302802 19338 consensus_queue.cc:260] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:40.304355 19338 ts_tablet_manager.cc:1428] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 22:33:40.319589 19343 tablet_bootstrap.cc:492] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c: Bootstrap starting.
I20260806 22:33:40.321314 19336 tablet_replica.cc:330] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91: stopping tablet replica
I20260806 22:33:40.321966 19336 raft_consensus.cc:2229] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:33:40.322856 19336 raft_consensus.cc:2258] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:40.323674 19334 tablet_replica.cc:330] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c: stopping tablet replica
I20260806 22:33:40.324388 19334 raft_consensus.cc:2229] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:33:40.324919 19334 raft_consensus.cc:2258] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:40.328228 19334 ts_tablet_manager.cc:1905] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:33:40.332260 19343 tablet_bootstrap.cc:654] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:40.334645 19344 tablet_bootstrap.cc:492] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91: Bootstrap starting.
I20260806 22:33:40.334993 19336 ts_tablet_manager.cc:1905] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:33:40.337716 19344 tablet_bootstrap.cc:654] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:40.339594 19333 tablet_replica.cc:330] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac: stopping tablet replica
I20260806 22:33:40.340288 19333 raft_consensus.cc:2229] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:33:40.340811 19333 raft_consensus.cc:2258] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:40.346146 19333 ts_tablet_manager.cc:1905] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:33:40.348901 19343 tablet_bootstrap.cc:492] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c: No bootstrap required, opened a new log
I20260806 22:33:40.349294 19343 ts_tablet_manager.cc:1397] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.007s
I20260806 22:33:40.350694 19343 raft_consensus.cc:348] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:40.351197 19343 raft_consensus.cc:374] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:40.351534 19343 raft_consensus.cc:729] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e73a6bb8c9d475aab217e3b5d81bd5c, State: Initialized, Role: FOLLOWER
I20260806 22:33:40.352070 19343 consensus_queue.cc:260] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:40.355376 19336 ts_tablet_manager.cc:1918] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:33:40.358054 19336 log.cc:1196] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/106201d85c5344b5bd80e4e3a27c831e
I20260806 22:33:40.358238 19343 ts_tablet_manager.cc:1428] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c: Time spent starting tablet: real 0.009s user 0.000s sys 0.003s
I20260806 22:33:40.359684 19336 ts_tablet_manager.cc:1939] T 106201d85c5344b5bd80e4e3a27c831e P 6cfe51d728f54515b0f5af9732db8b91: Deleting consensus metadata
I20260806 22:33:40.361227 19333 ts_tablet_manager.cc:1918] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:33:40.361482 19333 log.cc:1196] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/106201d85c5344b5bd80e4e3a27c831e
I20260806 22:33:40.362142 19333 ts_tablet_manager.cc:1939] T 106201d85c5344b5bd80e4e3a27c831e P c59ad9b6f91a4009901f247db3efc4ac: Deleting consensus metadata
I20260806 22:33:40.362818 18524 catalog_manager.cc:4841] TS 6cfe51d728f54515b0f5af9732db8b91 (127.15.198.130:34223): tablet 106201d85c5344b5bd80e4e3a27c831e (table test-table [id=fe4b26b80e824ff997e608880d76fddd]) successfully deleted
I20260806 22:33:40.364511 18529 catalog_manager.cc:4841] TS c59ad9b6f91a4009901f247db3efc4ac (127.15.198.131:44935): tablet 106201d85c5344b5bd80e4e3a27c831e (table test-table [id=fe4b26b80e824ff997e608880d76fddd]) successfully deleted
I20260806 22:33:40.365793 19344 tablet_bootstrap.cc:492] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91: No bootstrap required, opened a new log
I20260806 22:33:40.366108 19344 ts_tablet_manager.cc:1397] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91: Time spent bootstrapping tablet: real 0.032s user 0.004s sys 0.004s
I20260806 22:33:40.367532 19344 raft_consensus.cc:348] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:40.367990 19344 raft_consensus.cc:374] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:40.358294 19334 ts_tablet_manager.cc:1918] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:33:40.368410 19334 log.cc:1196] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/106201d85c5344b5bd80e4e3a27c831e
I20260806 22:33:40.369247 19334 ts_tablet_manager.cc:1939] T 106201d85c5344b5bd80e4e3a27c831e P 2e73a6bb8c9d475aab217e3b5d81bd5c: Deleting consensus metadata
I20260806 22:33:40.372115 18526 catalog_manager.cc:4841] TS 2e73a6bb8c9d475aab217e3b5d81bd5c (127.15.198.129:37659): tablet 106201d85c5344b5bd80e4e3a27c831e (table test-table [id=fe4b26b80e824ff997e608880d76fddd]) successfully deleted
I20260806 22:33:40.372763 19344 raft_consensus.cc:729] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cfe51d728f54515b0f5af9732db8b91, State: Initialized, Role: FOLLOWER
I20260806 22:33:40.373435 19344 consensus_queue.cc:260] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:40.374332 19344 ts_tablet_manager.cc:1428] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260806 22:33:40.591182 19259 raft_consensus.cc:482] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:33:40.591480 19261 raft_consensus.cc:482] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:33:40.591531 19259 raft_consensus.cc:504] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:40.591743 19261 raft_consensus.cc:504] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:40.592559 19261 leader_election.cc:288] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e73a6bb8c9d475aab217e3b5d81bd5c (127.15.198.129:37659), 6cfe51d728f54515b0f5af9732db8b91 (127.15.198.130:34223)
I20260806 22:33:40.592582 19259 leader_election.cc:288] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cfe51d728f54515b0f5af9732db8b91 (127.15.198.130:34223), c59ad9b6f91a4009901f247db3efc4ac (127.15.198.131:44935)
I20260806 22:33:40.593616 18763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f742ba066eb047e08ec26dd07f4d1aeb" candidate_uuid: "c59ad9b6f91a4009901f247db3efc4ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" is_pre_election: true
I20260806 22:33:40.593926 18763 raft_consensus.cc:2454] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c59ad9b6f91a4009901f247db3efc4ac in term 0.
I20260806 22:33:40.594961 18949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f742ba066eb047e08ec26dd07f4d1aeb" candidate_uuid: "c59ad9b6f91a4009901f247db3efc4ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfe51d728f54515b0f5af9732db8b91" is_pre_election: true
I20260806 22:33:40.595304 18949 raft_consensus.cc:2454] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c59ad9b6f91a4009901f247db3efc4ac in term 0.
I20260806 22:33:40.595389 19053 leader_election.cc:302] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e73a6bb8c9d475aab217e3b5d81bd5c, c59ad9b6f91a4009901f247db3efc4ac; no voters:
I20260806 22:33:40.594874 18950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f742ba066eb047e08ec26dd07f4d1aeb" candidate_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfe51d728f54515b0f5af9732db8b91" is_pre_election: true
I20260806 22:33:40.595855 18950 raft_consensus.cc:2454] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in term 0.
I20260806 22:33:40.596302 19155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f742ba066eb047e08ec26dd07f4d1aeb" candidate_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad9b6f91a4009901f247db3efc4ac" is_pre_election: true
I20260806 22:33:40.596637 19155 raft_consensus.cc:2454] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in term 0.
I20260806 22:33:40.596766 18672 leader_election.cc:302] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e73a6bb8c9d475aab217e3b5d81bd5c, 6cfe51d728f54515b0f5af9732db8b91; no voters:
I20260806 22:33:40.597632 19261 raft_consensus.cc:2781] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:33:40.597659 19259 raft_consensus.cc:2781] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:33:40.597765 19261 raft_consensus.cc:482] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:33:40.597785 19259 raft_consensus.cc:482] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:33:40.597896 19259 raft_consensus.cc:3037] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:40.597883 19261 raft_consensus.cc:3037] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:40.601311 19259 raft_consensus.cc:504] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:40.601760 19261 raft_consensus.cc:504] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:40.602890 18950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f742ba066eb047e08ec26dd07f4d1aeb" candidate_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfe51d728f54515b0f5af9732db8b91"
I20260806 22:33:40.603212 18950 raft_consensus.cc:3037] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:40.603472 18949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f742ba066eb047e08ec26dd07f4d1aeb" candidate_uuid: "c59ad9b6f91a4009901f247db3efc4ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfe51d728f54515b0f5af9732db8b91"
I20260806 22:33:40.604408 19261 leader_election.cc:288] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [CANDIDATE]: Term 1 election: Requested vote from peers 2e73a6bb8c9d475aab217e3b5d81bd5c (127.15.198.129:37659), 6cfe51d728f54515b0f5af9732db8b91 (127.15.198.130:34223)
I20260806 22:33:40.605824 19259 leader_election.cc:288] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [CANDIDATE]: Term 1 election: Requested vote from peers 6cfe51d728f54515b0f5af9732db8b91 (127.15.198.130:34223), c59ad9b6f91a4009901f247db3efc4ac (127.15.198.131:44935)
I20260806 22:33:40.606891 19155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f742ba066eb047e08ec26dd07f4d1aeb" candidate_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad9b6f91a4009901f247db3efc4ac"
I20260806 22:33:40.607024 18763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f742ba066eb047e08ec26dd07f4d1aeb" candidate_uuid: "c59ad9b6f91a4009901f247db3efc4ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c"
I20260806 22:33:40.607409 18763 raft_consensus.cc:2379] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c59ad9b6f91a4009901f247db3efc4ac in current term 1: Already voted for candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in this term.
I20260806 22:33:40.607405 19155 raft_consensus.cc:2379] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in current term 1: Already voted for candidate c59ad9b6f91a4009901f247db3efc4ac in this term.
I20260806 22:33:40.608021 19053 leader_election.cc:302] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c59ad9b6f91a4009901f247db3efc4ac; no voters: 2e73a6bb8c9d475aab217e3b5d81bd5c, 6cfe51d728f54515b0f5af9732db8b91
I20260806 22:33:40.609671 19261 raft_consensus.cc:2726] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 22:33:40.610146 18950 raft_consensus.cc:2454] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in term 1.
I20260806 22:33:40.610819 18672 leader_election.cc:302] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2e73a6bb8c9d475aab217e3b5d81bd5c, 6cfe51d728f54515b0f5af9732db8b91; no voters: c59ad9b6f91a4009901f247db3efc4ac
I20260806 22:33:40.611239 19259 raft_consensus.cc:2781] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:33:40.611523 19259 raft_consensus.cc:686] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 LEADER]: Becoming Leader. State: Replica: 2e73a6bb8c9d475aab217e3b5d81bd5c, State: Running, Role: LEADER
I20260806 22:33:40.612015 19259 consensus_queue.cc:237] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:40.616227 18546 catalog_manager.cc:5495] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e73a6bb8c9d475aab217e3b5d81bd5c (127.15.198.129). New cstate: current_term: 1 leader_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } health_report { overall_health: UNKNOWN } } }
I20260806 22:33:40.743842 19360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:40.737548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55070 (local address 127.15.198.129:37659)
0806 22:33:40.738189 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:40.738194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:40.738209 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:33:40.738326 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:40.738331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:40.738439 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:40.738608 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:40.739161 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:40.740169 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:40.741811 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:40.742259 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:40.742407 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:40.742875 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:40.743063 (+ 188us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:40.743168 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:40.743339 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 22:33:40.743451 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"spinlock_wait_cycles":9088,"thread_start_us":443,"threads_started":1}
I20260806 22:33:40.762928 19360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:40.755575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:48563 (local address 127.15.198.129:37659)
0806 22:33:40.755811 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:40.755822 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:40.756473 (+ 651us) server_negotiation.cc:407] Connection header received
0806 22:33:40.756712 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:40.756716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:40.756788 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:40.756865 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:40.757583 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:40.758583 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:40.761015 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:40.761416 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:40.761526 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:40.762116 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:40.762239 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:40.762312 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:40.762529 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 22:33:40.762607 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:33:40.763327 19365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:40.755562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:37659 (local address 127.15.198.129:48563)
0806 22:33:40.756308 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:40.756337 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:40.756545 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:40.756922 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:40.756929 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:40.756947 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:33:40.757380 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:40.757400 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:40.758725 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:40.758732 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:40.760812 (+ 2080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:40.760831 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:40.761635 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:40.761643 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:40.761849 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:40.761915 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:33:40.762359 (+ 444us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:33:40.762375 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 22:33:40.762486 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 22:33:40.762675 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":388,"threads_started":1}
I20260806 22:33:40.774253 18950 raft_consensus.cc:1261] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91 [term 1 FOLLOWER]: Refusing update from remote peer 2e73a6bb8c9d475aab217e3b5d81bd5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:33:40.774425 19155 raft_consensus.cc:1261] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Refusing update from remote peer 2e73a6bb8c9d475aab217e3b5d81bd5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:33:40.775096 19323 consensus_queue.cc:1035] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:33:40.775609 19259 consensus_queue.cc:1035] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:33:40.807730 19350 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 22:33:42.451198 18545 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:33:42.451692 18545 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:33:42.458571 18545 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f: Sending DeleteTablet for 3 replicas of tablet f742ba066eb047e08ec26dd07f4d1aeb
I20260806 22:33:42.460212 19122 tablet_service.cc:1514] Processing DeleteTablet for tablet f742ba066eb047e08ec26dd07f4d1aeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:42 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:49052
I20260806 22:33:42.460413 18920 tablet_service.cc:1514] Processing DeleteTablet for tablet f742ba066eb047e08ec26dd07f4d1aeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:42 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:51430
I20260806 22:33:42.460377 18733 tablet_service.cc:1514] Processing DeleteTablet for tablet f742ba066eb047e08ec26dd07f4d1aeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:42 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:55050
I20260806 22:33:42.474666 19398 tablet_replica.cc:330] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac: stopping tablet replica
I20260806 22:33:42.475281 19398 raft_consensus.cc:2229] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:33:42.475855 19398 raft_consensus.cc:2258] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:42.476800 19400 tablet_replica.cc:330] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91: stopping tablet replica
I20260806 22:33:42.477993 19399 tablet_replica.cc:330] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c: stopping tablet replica
I20260806 22:33:42.477937 19398 ts_tablet_manager.cc:1905] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:33:42.478302 19399 raft_consensus.cc:2229] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:33:42.478888 19400 raft_consensus.cc:2229] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:33:42.479445 19400 raft_consensus.cc:2258] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:42.480449 19399 raft_consensus.cc:2258] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:42.481691 19400 ts_tablet_manager.cc:1905] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:33:42.485585 19399 ts_tablet_manager.cc:1905] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 22:33:42 dist-test-slave-h3ft krb5kdc[18485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055622, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260806 22:33:42.574949 19400 ts_tablet_manager.cc:1918] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:33:42.574949 19399 ts_tablet_manager.cc:1918] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:33:42.574973 19398 ts_tablet_manager.cc:1918] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:33:42.575412 19399 log.cc:1196] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/f742ba066eb047e08ec26dd07f4d1aeb
I20260806 22:33:42.575443 19398 log.cc:1196] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/f742ba066eb047e08ec26dd07f4d1aeb
I20260806 22:33:42.575742 19400 log.cc:1196] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/f742ba066eb047e08ec26dd07f4d1aeb
I20260806 22:33:42.576131 19399 ts_tablet_manager.cc:1939] T f742ba066eb047e08ec26dd07f4d1aeb P 2e73a6bb8c9d475aab217e3b5d81bd5c: Deleting consensus metadata
I20260806 22:33:42.576131 19398 ts_tablet_manager.cc:1939] T f742ba066eb047e08ec26dd07f4d1aeb P c59ad9b6f91a4009901f247db3efc4ac: Deleting consensus metadata
I20260806 22:33:42.576444 19400 ts_tablet_manager.cc:1939] T f742ba066eb047e08ec26dd07f4d1aeb P 6cfe51d728f54515b0f5af9732db8b91: Deleting consensus metadata
W20260806 22:33:42.576848 16154 mini_kdc.cc:317] Time spent kinit for test-user: real 0.115s user 0.001s sys 0.024s
Aug 06 22:33:42 dist-test-slave-h3ft krb5kdc[18485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055622, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:42.607841 18529 catalog_manager.cc:4841] TS c59ad9b6f91a4009901f247db3efc4ac (127.15.198.131:44935): tablet f742ba066eb047e08ec26dd07f4d1aeb (table test-table [id=29174abda4544625b1d904306a858bee]) successfully deleted
I20260806 22:33:42.610023 18526 catalog_manager.cc:4841] TS 2e73a6bb8c9d475aab217e3b5d81bd5c (127.15.198.129:37659): tablet f742ba066eb047e08ec26dd07f4d1aeb (table test-table [id=29174abda4544625b1d904306a858bee]) successfully deleted
I20260806 22:33:42.610023 18524 catalog_manager.cc:4841] TS 6cfe51d728f54515b0f5af9732db8b91 (127.15.198.130:34223): tablet f742ba066eb047e08ec26dd07f4d1aeb (table test-table [id=29174abda4544625b1d904306a858bee]) successfully deleted
I20260806 22:33:42.616298 19410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:42.583320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43692 (local address 127.15.198.190:35053)
0806 22:33:42.584198 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:42.584220 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:42.584253 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:33:42.584417 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:42.584424 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:42.584580 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:42.584741 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:42.586954 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:42.588073 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:42.591784 (+ 3711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:42.592641 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:42.592860 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:42.604603 (+ 11743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:42.604613 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:42.604617 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:42.605562 (+ 945us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:42.611244 (+ 5682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:42.613065 (+ 1821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:42.613073 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:42.613076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:42.613344 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:42.614036 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:42.614042 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:42.614046 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:42.614550 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:42.614849 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:42.615546 (+ 697us) server_negotiation.cc:299] Negotiation successful
0806 22:33:42.615833 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":486,"threads_started":1}
I20260806 22:33:42.625413 18545 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:33:42.626762 18545 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:33:42.637705 18733 tablet_service.cc:1467] Processing CreateTablet for tablet e655c87ef0764180b5ce9967299bfef6 (DEFAULT_TABLE table=test-table [id=0b74b2c99be148848d6f509a8ad99480]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:42.638207 19122 tablet_service.cc:1467] Processing CreateTablet for tablet e655c87ef0764180b5ce9967299bfef6 (DEFAULT_TABLE table=test-table [id=0b74b2c99be148848d6f509a8ad99480]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:42.638587 18920 tablet_service.cc:1467] Processing CreateTablet for tablet e655c87ef0764180b5ce9967299bfef6 (DEFAULT_TABLE table=test-table [id=0b74b2c99be148848d6f509a8ad99480]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:42.638893 18733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e655c87ef0764180b5ce9967299bfef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:42.639243 19122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e655c87ef0764180b5ce9967299bfef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:42.639365 18920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e655c87ef0764180b5ce9967299bfef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:42.648877 19411 tablet_bootstrap.cc:492] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91: Bootstrap starting.
I20260806 22:33:42.652691 19414 tablet_bootstrap.cc:492] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac: Bootstrap starting.
I20260806 22:33:42.652428 19411 tablet_bootstrap.cc:654] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:42.653318 19412 tablet_bootstrap.cc:492] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Bootstrap starting.
I20260806 22:33:42.655814 19412 tablet_bootstrap.cc:654] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:42.655911 19414 tablet_bootstrap.cc:654] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:42.658730 19411 tablet_bootstrap.cc:492] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91: No bootstrap required, opened a new log
I20260806 22:33:42.659112 19411 ts_tablet_manager.cc:1397] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260806 22:33:42.659894 19412 tablet_bootstrap.cc:492] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c: No bootstrap required, opened a new log
I20260806 22:33:42.660159 19412 ts_tablet_manager.cc:1397] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 22:33:42.660413 19414 tablet_bootstrap.cc:492] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac: No bootstrap required, opened a new log
I20260806 22:33:42.660625 19414 ts_tablet_manager.cc:1397] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 22:33:42.660655 19411 raft_consensus.cc:348] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:42.661331 19411 raft_consensus.cc:374] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:42.661540 19411 raft_consensus.cc:729] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cfe51d728f54515b0f5af9732db8b91, State: Initialized, Role: FOLLOWER
I20260806 22:33:42.661468 19412 raft_consensus.cc:348] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:42.661988 19412 raft_consensus.cc:374] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:42.661710 19414 raft_consensus.cc:348] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:42.662020 19411 consensus_queue.cc:260] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:42.662256 19412 raft_consensus.cc:729] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e73a6bb8c9d475aab217e3b5d81bd5c, State: Initialized, Role: FOLLOWER
I20260806 22:33:42.662195 19414 raft_consensus.cc:374] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:42.662530 19414 raft_consensus.cc:729] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59ad9b6f91a4009901f247db3efc4ac, State: Initialized, Role: FOLLOWER
I20260806 22:33:42.662658 19412 consensus_queue.cc:260] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:42.663085 19414 consensus_queue.cc:260] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:42.663452 19412 ts_tablet_manager.cc:1428] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:33:42.664745 19411 ts_tablet_manager.cc:1428] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260806 22:33:42.667598 19414 ts_tablet_manager.cc:1428] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac: Time spent starting tablet: real 0.007s user 0.000s sys 0.005s
I20260806 22:33:42.949836 19383 raft_consensus.cc:482] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:33:42.950182 19383 raft_consensus.cc:504] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:42.951262 19383 leader_election.cc:288] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cfe51d728f54515b0f5af9732db8b91 (127.15.198.130:34223), c59ad9b6f91a4009901f247db3efc4ac (127.15.198.131:44935)
I20260806 22:33:42.952170 18950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e655c87ef0764180b5ce9967299bfef6" candidate_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfe51d728f54515b0f5af9732db8b91" is_pre_election: true
I20260806 22:33:42.952159 19155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e655c87ef0764180b5ce9967299bfef6" candidate_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad9b6f91a4009901f247db3efc4ac" is_pre_election: true
I20260806 22:33:42.952560 18950 raft_consensus.cc:2454] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in term 0.
I20260806 22:33:42.952625 19155 raft_consensus.cc:2454] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in term 0.
I20260806 22:33:42.953392 18675 leader_election.cc:302] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2e73a6bb8c9d475aab217e3b5d81bd5c, 6cfe51d728f54515b0f5af9732db8b91, c59ad9b6f91a4009901f247db3efc4ac; no voters:
I20260806 22:33:42.953845 19383 raft_consensus.cc:2781] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:33:42.954041 19383 raft_consensus.cc:482] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:33:42.954161 19383 raft_consensus.cc:3037] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:42.974654 19383 raft_consensus.cc:504] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:42.975795 19383 leader_election.cc:288] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [CANDIDATE]: Term 1 election: Requested vote from peers 6cfe51d728f54515b0f5af9732db8b91 (127.15.198.130:34223), c59ad9b6f91a4009901f247db3efc4ac (127.15.198.131:44935)
I20260806 22:33:42.976673 19155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e655c87ef0764180b5ce9967299bfef6" candidate_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad9b6f91a4009901f247db3efc4ac"
I20260806 22:33:42.976717 18950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e655c87ef0764180b5ce9967299bfef6" candidate_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfe51d728f54515b0f5af9732db8b91"
I20260806 22:33:42.977028 19155 raft_consensus.cc:3037] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:42.977052 18950 raft_consensus.cc:3037] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:43.078408 18950 raft_consensus.cc:2454] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in term 1.
I20260806 22:33:43.078382 19155 raft_consensus.cc:2454] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in term 1.
I20260806 22:33:43.079526 18672 leader_election.cc:302] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e73a6bb8c9d475aab217e3b5d81bd5c, 6cfe51d728f54515b0f5af9732db8b91; no voters:
I20260806 22:33:43.080066 19383 raft_consensus.cc:2781] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:33:43.080775 19383 raft_consensus.cc:686] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 LEADER]: Becoming Leader. State: Replica: 2e73a6bb8c9d475aab217e3b5d81bd5c, State: Running, Role: LEADER
I20260806 22:33:43.081225 19383 consensus_queue.cc:237] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:43.086222 18545 catalog_manager.cc:5495] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e73a6bb8c9d475aab217e3b5d81bd5c (127.15.198.129). New cstate: current_term: 1 leader_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } health_report { overall_health: UNKNOWN } } }
I20260806 22:33:43.148057 19439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:43.140657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55074 (local address 127.15.198.129:37659)
0806 22:33:43.141539 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:43.141550 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:43.141762 (+ 212us) server_negotiation.cc:407] Connection header received
0806 22:33:43.142083 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:43.142095 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:43.142180 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:43.142322 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:33:43.142941 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:43.143889 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:43.145528 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:43.146116 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:43.146302 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:43.147105 (+ 803us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:33:43.147266 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:33:43.147390 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:43.147547 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 22:33:43.147652 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":476,"threads_started":1}
I20260806 22:33:43.160236 19155 raft_consensus.cc:1261] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Refusing update from remote peer 2e73a6bb8c9d475aab217e3b5d81bd5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:33:43.160332 18950 raft_consensus.cc:1261] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91 [term 1 FOLLOWER]: Refusing update from remote peer 2e73a6bb8c9d475aab217e3b5d81bd5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:33:43.160990 19430 consensus_queue.cc:1035] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:33:43.161365 19383 consensus_queue.cc:1035] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:33:43.235462 18547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:33:43.235833 18547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:33:43.240667 18547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f: Sending DeleteTablet for 3 replicas of tablet e655c87ef0764180b5ce9967299bfef6
I20260806 22:33:43.242005 19122 tablet_service.cc:1514] Processing DeleteTablet for tablet e655c87ef0764180b5ce9967299bfef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:43 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:49052
I20260806 22:33:43.242141 18920 tablet_service.cc:1514] Processing DeleteTablet for tablet e655c87ef0764180b5ce9967299bfef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:43 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:51430
I20260806 22:33:43.242420 18733 tablet_service.cc:1514] Processing DeleteTablet for tablet e655c87ef0764180b5ce9967299bfef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:43 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:55050
I20260806 22:33:43.242951 19447 tablet_replica.cc:330] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac: stopping tablet replica
I20260806 22:33:43.243286 19447 raft_consensus.cc:2229] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:33:43.243395 19452 tablet_replica.cc:330] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c: stopping tablet replica
I20260806 22:33:43.243637 19447 raft_consensus.cc:2258] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:43.243649 19452 raft_consensus.cc:2229] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:33:43.243862 19448 tablet_replica.cc:330] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91: stopping tablet replica
I20260806 22:33:43.244180 19448 raft_consensus.cc:2229] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:33:43.244247 19452 raft_consensus.cc:2258] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:43.245008 19448 raft_consensus.cc:2258] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:43.244833 18547 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 22:33:43.245225 19447 ts_tablet_manager.cc:1905] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:33:43.245532 19452 ts_tablet_manager.cc:1905] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 22:33:43.246047 18547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:33:43.249410 19448 ts_tablet_manager.cc:1905] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:33:43.254102 19452 ts_tablet_manager.cc:1918] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:33:43.254164 19447 ts_tablet_manager.cc:1918] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:33:43.254345 19452 log.cc:1196] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/e655c87ef0764180b5ce9967299bfef6
I20260806 22:33:43.254415 19447 log.cc:1196] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/e655c87ef0764180b5ce9967299bfef6
I20260806 22:33:43.255115 19452 ts_tablet_manager.cc:1939] T e655c87ef0764180b5ce9967299bfef6 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Deleting consensus metadata
I20260806 22:33:43.255203 19447 ts_tablet_manager.cc:1939] T e655c87ef0764180b5ce9967299bfef6 P c59ad9b6f91a4009901f247db3efc4ac: Deleting consensus metadata
I20260806 22:33:43.256764 18733 tablet_service.cc:1467] Processing CreateTablet for tablet d42cc000f2314cd39f2d7c46dab62bb1 (DEFAULT_TABLE table=test-table [id=4e2da03a179a4484b5d721abf1ea4940]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:43.257121 19122 tablet_service.cc:1467] Processing CreateTablet for tablet d42cc000f2314cd39f2d7c46dab62bb1 (DEFAULT_TABLE table=test-table [id=4e2da03a179a4484b5d721abf1ea4940]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:43.257273 18920 tablet_service.cc:1467] Processing CreateTablet for tablet d42cc000f2314cd39f2d7c46dab62bb1 (DEFAULT_TABLE table=test-table [id=4e2da03a179a4484b5d721abf1ea4940]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:33:43.257457 18733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d42cc000f2314cd39f2d7c46dab62bb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:43.257774 19122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d42cc000f2314cd39f2d7c46dab62bb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:43.257886 18920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d42cc000f2314cd39f2d7c46dab62bb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:43.266976 18526 catalog_manager.cc:4841] TS 2e73a6bb8c9d475aab217e3b5d81bd5c (127.15.198.129:37659): tablet e655c87ef0764180b5ce9967299bfef6 (table test-table [id=0b74b2c99be148848d6f509a8ad99480]) successfully deleted
I20260806 22:33:43.267158 19454 tablet_bootstrap.cc:492] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Bootstrap starting.
I20260806 22:33:43.267467 18529 catalog_manager.cc:4841] TS c59ad9b6f91a4009901f247db3efc4ac (127.15.198.131:44935): tablet e655c87ef0764180b5ce9967299bfef6 (table test-table [id=0b74b2c99be148848d6f509a8ad99480]) successfully deleted
I20260806 22:33:43.269840 19456 tablet_bootstrap.cc:492] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91: Bootstrap starting.
I20260806 22:33:43.271032 19454 tablet_bootstrap.cc:654] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:43.271832 19455 tablet_bootstrap.cc:492] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac: Bootstrap starting.
I20260806 22:33:43.272068 19448 ts_tablet_manager.cc:1918] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:33:43.272249 19448 log.cc:1196] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/e655c87ef0764180b5ce9967299bfef6
I20260806 22:33:43.273221 19456 tablet_bootstrap.cc:654] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:43.273598 19448 ts_tablet_manager.cc:1939] T e655c87ef0764180b5ce9967299bfef6 P 6cfe51d728f54515b0f5af9732db8b91: Deleting consensus metadata
I20260806 22:33:43.274151 19455 tablet_bootstrap.cc:654] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:43.275179 18524 catalog_manager.cc:4841] TS 6cfe51d728f54515b0f5af9732db8b91 (127.15.198.130:34223): tablet e655c87ef0764180b5ce9967299bfef6 (table test-table [id=0b74b2c99be148848d6f509a8ad99480]) successfully deleted
I20260806 22:33:43.277747 19456 tablet_bootstrap.cc:492] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91: No bootstrap required, opened a new log
I20260806 22:33:43.277976 19456 ts_tablet_manager.cc:1397] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260806 22:33:43.279386 19456 raft_consensus.cc:348] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:43.279889 19456 raft_consensus.cc:374] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:43.280048 19456 raft_consensus.cc:729] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cfe51d728f54515b0f5af9732db8b91, State: Initialized, Role: FOLLOWER
I20260806 22:33:43.280465 19456 consensus_queue.cc:260] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:43.281488 19456 ts_tablet_manager.cc:1428] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260806 22:33:43.283612 19454 tablet_bootstrap.cc:492] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c: No bootstrap required, opened a new log
I20260806 22:33:43.283881 19454 ts_tablet_manager.cc:1397] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.007s
I20260806 22:33:43.284675 19455 tablet_bootstrap.cc:492] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac: No bootstrap required, opened a new log
I20260806 22:33:43.284948 19455 ts_tablet_manager.cc:1397] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.011s
I20260806 22:33:43.285056 19454 raft_consensus.cc:348] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:43.285437 19454 raft_consensus.cc:374] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:43.285569 19454 raft_consensus.cc:729] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e73a6bb8c9d475aab217e3b5d81bd5c, State: Initialized, Role: FOLLOWER
I20260806 22:33:43.285877 19454 consensus_queue.cc:260] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:43.286211 19455 raft_consensus.cc:348] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:43.286504 19455 raft_consensus.cc:374] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:43.286623 19454 ts_tablet_manager.cc:1428] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:33:43.286614 19455 raft_consensus.cc:729] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59ad9b6f91a4009901f247db3efc4ac, State: Initialized, Role: FOLLOWER
I20260806 22:33:43.286964 19455 consensus_queue.cc:260] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:43.287954 19455 ts_tablet_manager.cc:1428] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260806 22:33:43.618274 19430 raft_consensus.cc:482] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:33:43.618774 19430 raft_consensus.cc:504] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:43.620126 19430 leader_election.cc:288] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cfe51d728f54515b0f5af9732db8b91 (127.15.198.130:34223), c59ad9b6f91a4009901f247db3efc4ac (127.15.198.131:44935)
I20260806 22:33:43.621008 18950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42cc000f2314cd39f2d7c46dab62bb1" candidate_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfe51d728f54515b0f5af9732db8b91" is_pre_election: true
I20260806 22:33:43.621059 19151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42cc000f2314cd39f2d7c46dab62bb1" candidate_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad9b6f91a4009901f247db3efc4ac" is_pre_election: true
I20260806 22:33:43.621358 18950 raft_consensus.cc:2454] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in term 0.
I20260806 22:33:43.621448 19151 raft_consensus.cc:2454] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in term 0.
I20260806 22:33:43.622066 18672 leader_election.cc:302] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e73a6bb8c9d475aab217e3b5d81bd5c, 6cfe51d728f54515b0f5af9732db8b91; no voters:
I20260806 22:33:43.622488 19430 raft_consensus.cc:2781] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:33:43.622699 19430 raft_consensus.cc:482] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:33:43.622948 19430 raft_consensus.cc:3037] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:43.629415 19430 raft_consensus.cc:504] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:43.630409 19430 leader_election.cc:288] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [CANDIDATE]: Term 1 election: Requested vote from peers 6cfe51d728f54515b0f5af9732db8b91 (127.15.198.130:34223), c59ad9b6f91a4009901f247db3efc4ac (127.15.198.131:44935)
I20260806 22:33:43.631505 19151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42cc000f2314cd39f2d7c46dab62bb1" candidate_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad9b6f91a4009901f247db3efc4ac"
I20260806 22:33:43.631532 18950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42cc000f2314cd39f2d7c46dab62bb1" candidate_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfe51d728f54515b0f5af9732db8b91"
I20260806 22:33:43.631945 19151 raft_consensus.cc:3037] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:43.631959 18950 raft_consensus.cc:3037] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:43.635288 19151 raft_consensus.cc:2454] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in term 1.
I20260806 22:33:43.636230 18675 leader_election.cc:302] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e73a6bb8c9d475aab217e3b5d81bd5c, c59ad9b6f91a4009901f247db3efc4ac; no voters:
I20260806 22:33:43.636761 19430 raft_consensus.cc:2781] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:33:43.636888 18950 raft_consensus.cc:2454] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e73a6bb8c9d475aab217e3b5d81bd5c in term 1.
I20260806 22:33:43.637048 19430 raft_consensus.cc:686] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 LEADER]: Becoming Leader. State: Replica: 2e73a6bb8c9d475aab217e3b5d81bd5c, State: Running, Role: LEADER
I20260806 22:33:43.637426 19430 consensus_queue.cc:237] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } }
I20260806 22:33:43.641122 18545 catalog_manager.cc:5495] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e73a6bb8c9d475aab217e3b5d81bd5c (127.15.198.129). New cstate: current_term: 1 leader_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e73a6bb8c9d475aab217e3b5d81bd5c" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 } health_report { overall_health: UNKNOWN } } }
I20260806 22:33:43.759219 18950 raft_consensus.cc:1261] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91 [term 1 FOLLOWER]: Refusing update from remote peer 2e73a6bb8c9d475aab217e3b5d81bd5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:33:43.759613 19151 raft_consensus.cc:1261] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Refusing update from remote peer 2e73a6bb8c9d475aab217e3b5d81bd5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:33:43.760282 19422 consensus_queue.cc:1035] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfe51d728f54515b0f5af9732db8b91" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:33:43.761066 19464 consensus_queue.cc:1035] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c59ad9b6f91a4009901f247db3efc4ac" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:33:43.788539 19461 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 22:33:45.417796 18545 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:33:45.418306 18545 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:33:45.424316 18545 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 119ee579be3e4781ae9b556391f6a48f: Sending DeleteTablet for 3 replicas of tablet d42cc000f2314cd39f2d7c46dab62bb1
I20260806 22:33:45.425558 18733 tablet_service.cc:1514] Processing DeleteTablet for tablet d42cc000f2314cd39f2d7c46dab62bb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:45 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:55050
I20260806 22:33:45.425621 18920 tablet_service.cc:1514] Processing DeleteTablet for tablet d42cc000f2314cd39f2d7c46dab62bb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:45 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:51430
I20260806 22:33:45.425772 19122 tablet_service.cc:1514] Processing DeleteTablet for tablet d42cc000f2314cd39f2d7c46dab62bb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:33:45 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:49052
I20260806 22:33:45.426910 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 18649
I20260806 22:33:45.438628 19500 tablet_replica.cc:330] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c: stopping tablet replica
I20260806 22:33:45.439229 19500 raft_consensus.cc:2229] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:33:45.440331 19500 raft_consensus.cc:2258] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:45.442294 19500 ts_tablet_manager.cc:1905] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:33:45.443203 19503 tablet_replica.cc:330] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac: stopping tablet replica
I20260806 22:33:45.443688 19503 raft_consensus.cc:2229] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:33:45.444279 19503 raft_consensus.cc:2258] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:45.444984 19498 tablet_replica.cc:330] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91: stopping tablet replica
I20260806 22:33:45.445288 19498 raft_consensus.cc:2229] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:33:45.445688 19498 raft_consensus.cc:2258] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:33:45.446933 19498 ts_tablet_manager.cc:1905] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:33:45.446789 19503 ts_tablet_manager.cc:1905] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 22:33:45 dist-test-slave-h3ft krb5kdc[18485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055622, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:33:45.452876 19500 ts_tablet_manager.cc:1918] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:33:45.453127 19500 log.cc:1196] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/d42cc000f2314cd39f2d7c46dab62bb1
I20260806 22:33:45.453786 19500 ts_tablet_manager.cc:1939] T d42cc000f2314cd39f2d7c46dab62bb1 P 2e73a6bb8c9d475aab217e3b5d81bd5c: Deleting consensus metadata
I20260806 22:33:45.455747 18526 catalog_manager.cc:4841] TS 2e73a6bb8c9d475aab217e3b5d81bd5c (127.15.198.129:37659): tablet d42cc000f2314cd39f2d7c46dab62bb1 (table test-table [id=4e2da03a179a4484b5d721abf1ea4940]) successfully deleted
I20260806 22:33:45.456164 19498 ts_tablet_manager.cc:1918] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:33:45.456362 19498 log.cc:1196] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/d42cc000f2314cd39f2d7c46dab62bb1
I20260806 22:33:45.456445 19503 ts_tablet_manager.cc:1918] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:33:45.456631 19503 log.cc:1196] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/d42cc000f2314cd39f2d7c46dab62bb1
I20260806 22:33:45.456991 19498 ts_tablet_manager.cc:1939] T d42cc000f2314cd39f2d7c46dab62bb1 P 6cfe51d728f54515b0f5af9732db8b91: Deleting consensus metadata
I20260806 22:33:45.457182 19503 ts_tablet_manager.cc:1939] T d42cc000f2314cd39f2d7c46dab62bb1 P c59ad9b6f91a4009901f247db3efc4ac: Deleting consensus metadata
I20260806 22:33:45.458604 18524 catalog_manager.cc:4841] TS 6cfe51d728f54515b0f5af9732db8b91 (127.15.198.130:34223): tablet d42cc000f2314cd39f2d7c46dab62bb1 (table test-table [id=4e2da03a179a4484b5d721abf1ea4940]) successfully deleted
I20260806 22:33:45.458670 18529 catalog_manager.cc:4841] TS c59ad9b6f91a4009901f247db3efc4ac (127.15.198.131:44935): tablet d42cc000f2314cd39f2d7c46dab62bb1 (table test-table [id=4e2da03a179a4484b5d721abf1ea4940]) successfully deleted
I20260806 22:33:45.463841 19507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:45.430316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55078 (local address 127.15.198.129:37659)
0806 22:33:45.435530 (+ 5214us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:45.435542 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:45.435583 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:33:45.435683 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:45.435689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:45.435829 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:45.436001 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:45.439061 (+ 3060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:45.441673 (+ 2612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:45.444656 (+ 2983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:45.445563 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:45.445706 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:45.454161 (+ 8455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:45.454194 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:45.454226 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:45.454284 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:45.459806 (+ 5522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:45.460852 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:45.460863 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:45.460866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:45.461119 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:45.461547 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:45.461551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:45.461553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:45.462102 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:45.462556 (+ 454us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:45.462936 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 22:33:45.463310 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4975,"thread_start_us":4715,"threads_started":1}
W20260806 22:33:45.464748 18826 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55078
W20260806 22:33:45.465715 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 18649: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:33:45.466024 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 18649
I20260806 22:33:45.509979 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 18837
Aug 06 22:33:45 dist-test-slave-h3ft krb5kdc[18485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055622, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.198.130@KRBTEST.COM
I20260806 22:33:45.529863 19508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:45.510987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51446 (local address 127.15.198.130:34223)
0806 22:33:45.512138 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:45.512144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:45.512164 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:33:45.512238 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:45.512242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:45.512330 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:45.512464 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:45.514214 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:45.515202 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:45.516698 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:45.517212 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:45.517332 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:45.522658 (+ 5326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:45.522679 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:45.522701 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:45.522755 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:45.526269 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:45.527211 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:45.527221 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:45.527224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:45.527442 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:45.527912 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:45.527917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:45.527920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:45.528433 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:45.528815 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:45.529141 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 22:33:45.529432 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":959,"thread_start_us":696,"threads_started":1}
W20260806 22:33:45.530702 19016 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51446
W20260806 22:33:45.531543 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 18837: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:33:45.531735 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 18837
I20260806 22:33:45.571620 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 19033
Aug 06 22:33:45 dist-test-slave-h3ft krb5kdc[18485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055622, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.198.131@KRBTEST.COM
I20260806 22:33:45.592347 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:45.572560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49066 (local address 127.15.198.131:44935)
0806 22:33:45.573256 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:45.573261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:45.573282 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:33:45.573357 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:45.573362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:45.573451 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:45.573573 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:45.575358 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:45.576288 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:45.577739 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:45.578393 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:45.578508 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:45.584383 (+ 5875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:45.584410 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:45.584434 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:45.584488 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:45.588452 (+ 3964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:45.589481 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:45.589492 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:45.589496 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:45.589681 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:45.590189 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:45.590196 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:45.590199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:45.590718 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:45.591214 (+ 496us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:45.591567 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 22:33:45.591894 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":313,"threads_started":1}
W20260806 22:33:45.593493 19214 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49066
W20260806 22:33:45.594442 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 19033: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:33:45.594668 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 19033
I20260806 22:33:45.630577 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 18504
I20260806 22:33:45.632598 18605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:45.786286 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 18504
2026-08-06T22:33:45Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (11273 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:33:45 dist-test-slave-h3ft krb5kdc[19514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:33:45 dist-test-slave-h3ft krb5kdc[19514](info): set up 2 sockets
Aug 06 22:33:45 dist-test-slave-h3ft krb5kdc[19514](info): commencing operation
krb5kdc: starting...
W20260806 22:33:47.229754 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.370s user 0.002s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:33:47 dist-test-slave-h3ft krb5kdc[19514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:33:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:33:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:33:47.533420 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:39053
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:46311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:39053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:47.969568 19532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:47.970038 19532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:47.970148 19532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:47.979700 19532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:33:47.979887 19532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:47.979941 19532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:47.980006 19532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:33:47.980046 19532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:33:47.991684 19532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:46311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:39053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:39053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:47.993620 19532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:47.995404 19532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:48.010963 19539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:48.010948 19545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:48.011722 19542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:48.013039 19532 server_base.cc:1034] running on GCE node
I20260806 22:33:48.014249 19532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:48.016341 19532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:48.017635 19532 hybrid_clock.cc:648] HybridClock initialized: now 1786055628017555 us; error 75 us; skew 500 ppm
I20260806 22:33:48.018143 19532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:48 dist-test-slave-h3ft krb5kdc[19514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055628, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:48.026027 19532 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:33:48.028210 19532 webserver.cc:460] Webserver started at http://127.15.198.190:43813/ using document root <none> and password file <none>
I20260806 22:33:48.029165 19532 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:48.029318 19532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:48.029742 19532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:48.033725 19532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "fadd9c360e4c4214856aa3d395e1a40e"
format_stamp: "Formatted at 2026-08-06 22:33:48 on dist-test-slave-h3ft"
server_key: "a1d694b58c6384289955413226e4e047"
server_key_iv: "ed3ca40488ea9df6e317f25c246d9216"
server_key_version: "encryptionkey@0"
I20260806 22:33:48.034670 19532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "fadd9c360e4c4214856aa3d395e1a40e"
format_stamp: "Formatted at 2026-08-06 22:33:48 on dist-test-slave-h3ft"
server_key: "a1d694b58c6384289955413226e4e047"
server_key_iv: "ed3ca40488ea9df6e317f25c246d9216"
server_key_version: "encryptionkey@0"
I20260806 22:33:48.041846 19532 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260806 22:33:48.047088 19553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:48.048929 19532 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 22:33:48.049139 19532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "fadd9c360e4c4214856aa3d395e1a40e"
format_stamp: "Formatted at 2026-08-06 22:33:48 on dist-test-slave-h3ft"
server_key: "a1d694b58c6384289955413226e4e047"
server_key_iv: "ed3ca40488ea9df6e317f25c246d9216"
server_key_version: "encryptionkey@0"
I20260806 22:33:48.049361 19532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:48.065316 19532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:48.071148 19532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:48.071460 19532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:48.095789 19532 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:39053
I20260806 22:33:48.095814 19623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:39053 every 8 connection(s)
I20260806 22:33:48.097769 19532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:33:48.103456 19626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:48.107923 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 19532
I20260806 22:33:48.108224 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:33:48.109172 16154 external_mini_cluster.cc:1428] Setting key 8bfcbe9fa649ae02b37f6b180cceca6d
I20260806 22:33:48.120786 19626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e: Bootstrap starting.
I20260806 22:33:48.125957 19626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:48.127874 19626 log.cc:824] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e: Log is configured to *not* fsync() on all Append() calls
Aug 06 22:33:48 dist-test-slave-h3ft krb5kdc[19514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:48.132877 19626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e: No bootstrap required, opened a new log
I20260806 22:33:48.142551 19631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:48.112240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53592 (local address 127.15.198.190:39053)
0806 22:33:48.113370 (+ 1130us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:48.113386 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:48.113445 (+ 59us) server_negotiation.cc:407] Connection header received
0806 22:33:48.115714 (+ 2269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:48.115750 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:48.116385 (+ 635us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:48.116873 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:48.119237 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:48.121112 (+ 1875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:48.123172 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:48.123987 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:48.124154 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:48.130546 (+ 6392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:48.130559 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:48.130588 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:48.130643 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:48.135764 (+ 5121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:48.136907 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:48.136926 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:48.136939 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:48.137227 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:48.138122 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:48.138128 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:48.138132 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:48.139127 (+ 995us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:48.139531 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:48.140018 (+ 487us) server_negotiation.cc:299] Negotiation successful
0806 22:33:48.140652 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":850,"thread_start_us":537,"threads_started":1}
I20260806 22:33:48.144641 19626 raft_consensus.cc:348] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fadd9c360e4c4214856aa3d395e1a40e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 39053 } }
I20260806 22:33:48.145115 19626 raft_consensus.cc:374] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:48.145239 19626 raft_consensus.cc:729] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fadd9c360e4c4214856aa3d395e1a40e, State: Initialized, Role: FOLLOWER
I20260806 22:33:48.145955 19626 consensus_queue.cc:260] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fadd9c360e4c4214856aa3d395e1a40e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 39053 } }
I20260806 22:33:48.146250 19626 raft_consensus.cc:388] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:33:48.146437 19626 raft_consensus.cc:482] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:33:48.146657 19626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:48.149400 19626 raft_consensus.cc:504] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fadd9c360e4c4214856aa3d395e1a40e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 39053 } }
I20260806 22:33:48.150063 19626 leader_election.cc:302] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fadd9c360e4c4214856aa3d395e1a40e; no voters:
I20260806 22:33:48.150838 19626 leader_election.cc:288] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:33:48.151048 19633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:33:48.152338 19633 raft_consensus.cc:686] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [term 1 LEADER]: Becoming Leader. State: Replica: fadd9c360e4c4214856aa3d395e1a40e, State: Running, Role: LEADER
I20260806 22:33:48.152945 19633 consensus_queue.cc:237] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fadd9c360e4c4214856aa3d395e1a40e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 39053 } }
I20260806 22:33:48.153673 19626 sys_catalog.cc:564] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:33:48.156873 19636 sys_catalog.cc:455] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [sys.catalog]: SysCatalogTable state changed. Reason: New leader fadd9c360e4c4214856aa3d395e1a40e. Latest consensus state: current_term: 1 leader_uuid: "fadd9c360e4c4214856aa3d395e1a40e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fadd9c360e4c4214856aa3d395e1a40e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 39053 } } }
I20260806 22:33:48.156862 19635 sys_catalog.cc:455] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fadd9c360e4c4214856aa3d395e1a40e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fadd9c360e4c4214856aa3d395e1a40e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 39053 } } }
I20260806 22:33:48.157474 19635 sys_catalog.cc:458] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [sys.catalog]: This master's current role is: LEADER
I20260806 22:33:48.157470 19636 sys_catalog.cc:458] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e [sys.catalog]: This master's current role is: LEADER
I20260806 22:33:48.161978 19646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:33:48.168057 19646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:33:48.179987 19646 catalog_manager.cc:1347] Generated new cluster ID: 37258935397e47e88a6b7284ccc3071a
I20260806 22:33:48.180199 19646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:33:48.227304 19646 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:33:48.229125 19646 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:33:48.237124 19646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fadd9c360e4c4214856aa3d395e1a40e: Generated new TSK 0
I20260806 22:33:48.238431 19646 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
I20260806 22:33:48.328665 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:39053
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:46311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:48.792342 19661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:48.792778 19661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:48.792974 19661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:48.801349 19661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:48.801528 19661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:48.801748 19661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:33:48.814656 19661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:46311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:39053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:48.816573 19661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:48.818394 19661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:48.837379 19668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:48.837282 19669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:48.837283 19672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:48.838631 19661 server_base.cc:1034] running on GCE node
I20260806 22:33:48.839607 19661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:48.840924 19661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:48.842192 19661 hybrid_clock.cc:648] HybridClock initialized: now 1786055628842114 us; error 75 us; skew 500 ppm
I20260806 22:33:48.842681 19661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:48 dist-test-slave-h3ft krb5kdc[19514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055628, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:48.848522 19661 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:33:48.851150 19661 webserver.cc:460] Webserver started at http://127.15.198.129:36865/ using document root <none> and password file <none>
I20260806 22:33:48.852120 19661 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:48.852263 19661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:48.852702 19661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:48.857147 19661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "376c636f0c694c398fef39acc941ba46"
format_stamp: "Formatted at 2026-08-06 22:33:48 on dist-test-slave-h3ft"
server_key: "717390bf59f90a4a1250b71c19b5e535"
server_key_iv: "182aef2ea8102e26603bfc94d969820a"
server_key_version: "encryptionkey@0"
I20260806 22:33:48.858333 19661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "376c636f0c694c398fef39acc941ba46"
format_stamp: "Formatted at 2026-08-06 22:33:48 on dist-test-slave-h3ft"
server_key: "717390bf59f90a4a1250b71c19b5e535"
server_key_iv: "182aef2ea8102e26603bfc94d969820a"
server_key_version: "encryptionkey@0"
I20260806 22:33:48.865397 19661 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 22:33:48.870599 19682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:48.872990 19661 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 22:33:48.873339 19661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "376c636f0c694c398fef39acc941ba46"
format_stamp: "Formatted at 2026-08-06 22:33:48 on dist-test-slave-h3ft"
server_key: "717390bf59f90a4a1250b71c19b5e535"
server_key_iv: "182aef2ea8102e26603bfc94d969820a"
server_key_version: "encryptionkey@0"
I20260806 22:33:48.873665 19661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:48.894969 19661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:48.900867 19661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:48.901376 19661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:48.902889 19661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:33:48.904888 19661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:48.904987 19661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:48.905099 19661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:48.905139 19661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:48.947255 19661 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:37183
I20260806 22:33:48.947292 19842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:37183 every 8 connection(s)
I20260806 22:33:48.949039 19661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:33:48.955852 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 19661
I20260806 22:33:48.956179 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:33:48.957016 16154 external_mini_cluster.cc:1428] Setting key 5b59ba9573d32060387a9d36339fcf1f
Aug 06 22:33:48 dist-test-slave-h3ft krb5kdc[19514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055628, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:48.977082 19849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:48.952640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:57247 (local address 127.15.198.190:39053)
0806 22:33:48.953426 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:48.953434 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:48.954855 (+ 1421us) server_negotiation.cc:407] Connection header received
0806 22:33:48.957715 (+ 2860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:48.957723 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:48.957871 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:48.958037 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:48.960739 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:48.961685 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:48.963846 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:48.964346 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:48.964530 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:48.969709 (+ 5179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:48.969715 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:48.969719 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:48.969770 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:48.972979 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:48.974035 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:48.974040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:48.974043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:48.974194 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:48.974743 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:48.974747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:48.974749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:48.975185 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:48.975367 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:48.976314 (+ 947us) server_negotiation.cc:299] Negotiation successful
0806 22:33:48.976614 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":402,"threads_started":1}
I20260806 22:33:48.978622 19850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:48.953182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:39053 (local address 127.15.198.129:57247)
0806 22:33:48.954583 (+ 1401us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:48.954639 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:48.957263 (+ 2624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:48.958476 (+ 1213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:48.958494 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:48.959577 (+ 1083us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:48.960499 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:48.960526 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:48.961913 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:48.961919 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:48.963646 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:48.963663 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:48.964682 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:48.964693 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:48.965023 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:48.966124 (+ 1101us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:48.966161 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:48.969284 (+ 3123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:48.973206 (+ 3922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:48.973219 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:48.973252 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:48.973840 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:48.974326 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:48.974331 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:48.974335 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:48.974593 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:48.975334 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:48.975345 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:48.975849 (+ 504us) client_negotiation.cc:769] Sending connection context
0806 22:33:48.976192 (+ 343us) client_negotiation.cc:240] Negotiation successful
0806 22:33:48.976939 (+ 747us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"thread_start_us":488,"threads_started":1}
I20260806 22:33:48.981118 19843 heartbeater.cc:347] Connected to a master server at 127.15.198.190:39053
I20260806 22:33:48.981534 19843 heartbeater.cc:464] Registering TS with master...
I20260806 22:33:48.982549 19843 heartbeater.cc:511] Master 127.15.198.190:39053 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
I20260806 22:33:48.985532 19578 ts_manager.cc:194] Registered new tserver with Master: 376c636f0c694c398fef39acc941ba46 (127.15.198.129:37183)
I20260806 22:33:48.987749 19578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:57247
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
I20260806 22:33:49.029775 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:39053
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:46311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:49.468180 19854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:49.468614 19854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:49.468806 19854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:49.476980 19854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:49.477182 19854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:49.477432 19854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:33:49.488950 19854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:46311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:39053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:49.490830 19854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:49.492717 19854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:49.510303 19861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:49.510329 19860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:49.510303 19865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:49.511083 19854 server_base.cc:1034] running on GCE node
I20260806 22:33:49.512032 19854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:49.513314 19854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:49.514544 19854 hybrid_clock.cc:648] HybridClock initialized: now 1786055629514475 us; error 49 us; skew 500 ppm
I20260806 22:33:49.514945 19854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:49 dist-test-slave-h3ft krb5kdc[19514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055629, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:49.518810 19854 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:33:49.520809 19854 webserver.cc:460] Webserver started at http://127.15.198.130:35701/ using document root <none> and password file <none>
I20260806 22:33:49.521715 19854 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:49.521881 19854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:49.522349 19854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:49.526160 19854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "c6b6869189d7439eab8770bfa14e5d82"
format_stamp: "Formatted at 2026-08-06 22:33:49 on dist-test-slave-h3ft"
server_key: "8aae36e91e0fa1c5a557fff13673434b"
server_key_iv: "2ad714a92c026dc02051d50fc78e4bef"
server_key_version: "encryptionkey@0"
I20260806 22:33:49.527176 19854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "c6b6869189d7439eab8770bfa14e5d82"
format_stamp: "Formatted at 2026-08-06 22:33:49 on dist-test-slave-h3ft"
server_key: "8aae36e91e0fa1c5a557fff13673434b"
server_key_iv: "2ad714a92c026dc02051d50fc78e4bef"
server_key_version: "encryptionkey@0"
I20260806 22:33:49.533447 19854 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 22:33:49.537621 19876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:49.539189 19854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:33:49.539402 19854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "c6b6869189d7439eab8770bfa14e5d82"
format_stamp: "Formatted at 2026-08-06 22:33:49 on dist-test-slave-h3ft"
server_key: "8aae36e91e0fa1c5a557fff13673434b"
server_key_iv: "2ad714a92c026dc02051d50fc78e4bef"
server_key_version: "encryptionkey@0"
I20260806 22:33:49.539645 19854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:49.561076 19854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:49.566198 19854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:49.566556 19854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:49.567890 19854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:33:49.569720 19854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:49.569837 19854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:49.570017 19854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:49.570096 19854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:49.610210 19854 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:46159
I20260806 22:33:49.610263 20043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:46159 every 8 connection(s)
I20260806 22:33:49.611927 19854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:33:49.614619 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 19854
I20260806 22:33:49.614854 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:33:49.615585 16154 external_mini_cluster.cc:1428] Setting key a0841cc334258bef8f7dd5db1c596961
Aug 06 22:33:49 dist-test-slave-h3ft krb5kdc[19514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055629, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:49.639321 20048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:49.615515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:50571 (local address 127.15.198.190:39053)
0806 22:33:49.616254 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:49.616259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:49.617505 (+ 1246us) server_negotiation.cc:407] Connection header received
0806 22:33:49.619830 (+ 2325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:49.619836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:49.619943 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:49.620085 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:49.621996 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:49.622797 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:49.624584 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:49.625091 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:49.625242 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:49.630251 (+ 5009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:49.630262 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:49.630272 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:49.630333 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:49.634926 (+ 4593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:49.635893 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:49.635903 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:49.635910 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:49.636114 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:49.636626 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:49.636634 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:49.636641 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:49.637129 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:49.637311 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:49.638361 (+ 1050us) server_negotiation.cc:299] Negotiation successful
0806 22:33:49.638708 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":464,"threads_started":1}
I20260806 22:33:49.640420 20049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:49.616272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:39053 (local address 127.15.198.130:50571)
0806 22:33:49.617324 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:49.617371 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:49.619497 (+ 2126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:49.620385 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:49.620397 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:49.621042 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:49.621781 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:49.621803 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:49.622983 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:49.622988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:49.624416 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:49.624429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:49.625361 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:49.625369 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:49.625660 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:49.626585 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:49.626621 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:49.629908 (+ 3287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:49.635147 (+ 5239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:49.635156 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:49.635176 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:49.635714 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:49.636247 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:49.636250 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:49.636253 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:49.636471 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:49.637312 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:49.637320 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:49.637814 (+ 494us) client_negotiation.cc:769] Sending connection context
0806 22:33:49.638176 (+ 362us) client_negotiation.cc:240] Negotiation successful
0806 22:33:49.638724 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"thread_start_us":393,"threads_started":1}
I20260806 22:33:49.642939 20045 heartbeater.cc:347] Connected to a master server at 127.15.198.190:39053
I20260806 22:33:49.643499 20045 heartbeater.cc:464] Registering TS with master...
I20260806 22:33:49.644904 20045 heartbeater.cc:511] Master 127.15.198.190:39053 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
I20260806 22:33:49.648016 19578 ts_manager.cc:194] Registered new tserver with Master: c6b6869189d7439eab8770bfa14e5d82 (127.15.198.130:46159)
I20260806 22:33:49.649173 19578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:50571
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
I20260806 22:33:49.689312 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:39053
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:46311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:33:49.992178 19843 heartbeater.cc:503] Master 127.15.198.190:39053 was elected leader, sending a full tablet report...
W20260806 22:33:50.144052 20054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:50.144464 20054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:50.144647 20054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:50.154186 20054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:50.154421 20054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:50.154737 20054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:33:50.167910 20054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:46311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:39053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:50.169843 20054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:50.171667 20054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:50.189769 20062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:50.189831 20064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:50.189734 20067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:50.197589 20054 server_base.cc:1034] running on GCE node
I20260806 22:33:50.198735 20054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:50.200309 20054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:50.201710 20054 hybrid_clock.cc:648] HybridClock initialized: now 1786055630201581 us; error 88 us; skew 500 ppm
I20260806 22:33:50.202319 20054 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:50 dist-test-slave-h3ft krb5kdc[19514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055630, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:50.207700 20054 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:33:50.210079 20054 webserver.cc:460] Webserver started at http://127.15.198.131:45803/ using document root <none> and password file <none>
I20260806 22:33:50.211293 20054 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:50.211505 20054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:50.211946 20054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:50.216248 20054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "87e71c45f71c4d4fa47d54640dd4d111"
format_stamp: "Formatted at 2026-08-06 22:33:50 on dist-test-slave-h3ft"
server_key: "955381f70e64867ba5a89795049c7b52"
server_key_iv: "1a1d7292a0a2fbe42dda95371bddec30"
server_key_version: "encryptionkey@0"
I20260806 22:33:50.217311 20054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "87e71c45f71c4d4fa47d54640dd4d111"
format_stamp: "Formatted at 2026-08-06 22:33:50 on dist-test-slave-h3ft"
server_key: "955381f70e64867ba5a89795049c7b52"
server_key_iv: "1a1d7292a0a2fbe42dda95371bddec30"
server_key_version: "encryptionkey@0"
I20260806 22:33:50.224836 20054 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 22:33:50.230213 20076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:50.232771 20054 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260806 22:33:50.233135 20054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "87e71c45f71c4d4fa47d54640dd4d111"
format_stamp: "Formatted at 2026-08-06 22:33:50 on dist-test-slave-h3ft"
server_key: "955381f70e64867ba5a89795049c7b52"
server_key_iv: "1a1d7292a0a2fbe42dda95371bddec30"
server_key_version: "encryptionkey@0"
I20260806 22:33:50.233455 20054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:50.252144 20054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:50.258651 20054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:50.259163 20054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:50.260578 20054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:33:50.262516 20054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:50.262648 20054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:50.262835 20054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:50.262930 20054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:50.306751 20054 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:43573
I20260806 22:33:50.306792 20240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:43573 every 8 connection(s)
I20260806 22:33:50.308609 20054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:33:50.316408 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 20054
I20260806 22:33:50.316649 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:33:50.317214 16154 external_mini_cluster.cc:1428] Setting key bf79abdd244eac518f82bdbf2eb65178
Aug 06 22:33:50 dist-test-slave-h3ft krb5kdc[19514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055630, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:50.335249 20244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:50.312085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:56581 (local address 127.15.198.190:39053)
0806 22:33:50.312762 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:50.312767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:50.313957 (+ 1190us) server_negotiation.cc:407] Connection header received
0806 22:33:50.316364 (+ 2407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:50.316371 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:50.316491 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:50.316658 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:50.319125 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:50.319991 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:50.322055 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:50.322521 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:50.322701 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:50.326901 (+ 4200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:50.326907 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:50.326910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:50.326953 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:50.330435 (+ 3482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:50.331850 (+ 1415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:50.331858 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:50.331864 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:50.332052 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:50.332708 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:50.332713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:50.332715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:50.333088 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:50.333302 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:50.334345 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0806 22:33:50.334664 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":328,"threads_started":1}
I20260806 22:33:50.336683 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:50.312789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:39053 (local address 127.15.198.131:56581)
0806 22:33:50.313727 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:50.313786 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:50.316007 (+ 2221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:50.317167 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:50.317178 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:50.317906 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:50.318868 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:50.318894 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:50.320183 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:50.320188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:50.321881 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:50.321895 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:50.322833 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:50.322841 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:50.323189 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:50.323979 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:50.324012 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:50.326639 (+ 2627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:50.330945 (+ 4306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:50.330957 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:50.330978 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:50.331666 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:50.332211 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:50.332217 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:50.332222 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:50.332530 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:50.333257 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:50.333268 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:50.333803 (+ 535us) client_negotiation.cc:769] Sending connection context
0806 22:33:50.334186 (+ 383us) client_negotiation.cc:240] Negotiation successful
0806 22:33:50.334934 (+ 748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":348,"threads_started":1}
I20260806 22:33:50.338750 20241 heartbeater.cc:347] Connected to a master server at 127.15.198.190:39053
I20260806 22:33:50.339263 20241 heartbeater.cc:464] Registering TS with master...
I20260806 22:33:50.340312 20241 heartbeater.cc:511] Master 127.15.198.190:39053 requested a full tablet report, sending...
I20260806 22:33:50.342276 19578 ts_manager.cc:194] Registered new tserver with Master: 87e71c45f71c4d4fa47d54640dd4d111 (127.15.198.131:43573)
I20260806 22:33:50.343393 19578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:56581
I20260806 22:33:50.347157 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:33:50.398705 20244 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:33:50.395347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41322 (local address 127.15.198.190:39053)
0806 22:33:50.395605 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:50.395611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:50.396217 (+ 606us) server_negotiation.cc:407] Connection header received
0806 22:33:50.396415 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:50.396420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:50.396563 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:50.396677 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:50.398293 (+ 1616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41322: BlockingRecv error: recv got EOF from 127.0.0.1:41322 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 22:33:50.402372 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 19661
W20260806 22:33:50.406335 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 19661: Not authorized: Client connection negotiation failed: client connection to 127.15.198.129:37183: server requires authentication, but client does not have Kerberos credentials available
I20260806 22:33:50.406663 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 19661
I20260806 22:33:50.435840 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 19854
W20260806 22:33:50.439575 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 19854: Not authorized: Client connection negotiation failed: client connection to 127.15.198.130:46159: server requires authentication, but client does not have Kerberos credentials available
I20260806 22:33:50.439736 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 19854
I20260806 22:33:50.474691 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 20054
W20260806 22:33:50.478833 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 20054: Not authorized: Client connection negotiation failed: client connection to 127.15.198.131:43573: server requires authentication, but client does not have Kerberos credentials available
I20260806 22:33:50.479265 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 20054
W20260806 22:33:50.479293 20259 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:33:50.476187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59606 (local address 127.15.198.131:43573)
0806 22:33:50.476966 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:50.476974 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:50.477022 (+ 48us) server_negotiation.cc:407] Connection header received
0806 22:33:50.477103 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:50.477112 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:50.477398 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:50.477644 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:50.478766 (+ 1122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59606: BlockingRecv error: recv got EOF from 127.0.0.1:59606 (error 108)
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":386,"threads_started":1}
I20260806 22:33:50.508579 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 19532
I20260806 22:33:50.510337 19619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:50.631507 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 19532
2026-08-06T22:33:50Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4831 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:33:50 dist-test-slave-h3ft krb5kdc[20264](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:33:50 dist-test-slave-h3ft krb5kdc[20264](info): set up 2 sockets
Aug 06 22:33:50 dist-test-slave-h3ft krb5kdc[20264](info): commencing operation
krb5kdc: starting...
W20260806 22:33:52.021422 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:33:52 dist-test-slave-h3ft krb5kdc[20264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:33:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:33:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:33:52.320994 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:33241
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:45579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:33241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:52.727198 20280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:52.727589 20280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:52.727692 20280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:52.735865 20280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:33:52.736030 20280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:52.736084 20280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:52.736147 20280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:33:52.736202 20280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:33:52.748224 20280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:45579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:33241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:33241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:52.750214 20280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:52.751982 20280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:52.767215 20288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:52.767246 20287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:52.767494 20290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:52.768718 20280 server_base.cc:1034] running on GCE node
I20260806 22:33:52.769876 20280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:52.771803 20280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:52.773046 20280 hybrid_clock.cc:648] HybridClock initialized: now 1786055632772988 us; error 49 us; skew 500 ppm
I20260806 22:33:52.773523 20280 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:52 dist-test-slave-h3ft krb5kdc[20264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055632, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:52.777985 20280 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:33:52.780613 20280 webserver.cc:460] Webserver started at http://127.15.198.190:38711/ using document root <none> and password file <none>
I20260806 22:33:52.781615 20280 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:52.781803 20280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:52.782246 20280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:52.786680 20280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "c7baa9daa22a4bee90b1af94d37ba405"
format_stamp: "Formatted at 2026-08-06 22:33:52 on dist-test-slave-h3ft"
server_key: "f334dc918c08e9cf9edb2cca4e6ee34e"
server_key_iv: "05485c9758c399ffa9a2c9662bc451e8"
server_key_version: "encryptionkey@0"
I20260806 22:33:52.788048 20280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "c7baa9daa22a4bee90b1af94d37ba405"
format_stamp: "Formatted at 2026-08-06 22:33:52 on dist-test-slave-h3ft"
server_key: "f334dc918c08e9cf9edb2cca4e6ee34e"
server_key_iv: "05485c9758c399ffa9a2c9662bc451e8"
server_key_version: "encryptionkey@0"
I20260806 22:33:52.795624 20280 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260806 22:33:52.800305 20301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:52.802213 20280 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260806 22:33:52.802562 20280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "c7baa9daa22a4bee90b1af94d37ba405"
format_stamp: "Formatted at 2026-08-06 22:33:52 on dist-test-slave-h3ft"
server_key: "f334dc918c08e9cf9edb2cca4e6ee34e"
server_key_iv: "05485c9758c399ffa9a2c9662bc451e8"
server_key_version: "encryptionkey@0"
I20260806 22:33:52.802831 20280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:52.823817 20280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:52.829003 20280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:52.829372 20280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:52.853163 20280 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:33241
I20260806 22:33:52.853200 20373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:33241 every 8 connection(s)
I20260806 22:33:52.854991 20280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:33:52.860014 20375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:33:52.862613 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 20280
I20260806 22:33:52.862939 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:33:52.863731 16154 external_mini_cluster.cc:1428] Setting key d91ef6bba622c3e5b4f106e06444c964
I20260806 22:33:52.876647 20375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405: Bootstrap starting.
I20260806 22:33:52.881369 20375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405: Neither blocks nor log segments found. Creating new log.
Aug 06 22:33:52 dist-test-slave-h3ft krb5kdc[20264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:52.882820 20375 log.cc:824] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405: Log is configured to *not* fsync() on all Append() calls
I20260806 22:33:52.886837 20375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405: No bootstrap required, opened a new log
I20260806 22:33:52.892469 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:52.866252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48996 (local address 127.15.198.190:33241)
0806 22:33:52.867383 (+ 1131us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:52.867404 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:52.867471 (+ 67us) server_negotiation.cc:407] Connection header received
0806 22:33:52.870125 (+ 2654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:52.870168 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:52.870864 (+ 696us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:52.871525 (+ 661us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:52.873277 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:52.875574 (+ 2297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:52.877045 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:52.878225 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:52.878433 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:52.883234 (+ 4801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:52.883245 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:52.883267 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:52.883321 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:52.887072 (+ 3751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:52.887988 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:52.887998 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:52.888007 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:52.888240 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:52.888725 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:52.888731 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:52.888734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:52.889449 (+ 715us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:52.889865 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:52.890447 (+ 582us) server_negotiation.cc:299] Negotiation successful
0806 22:33:52.891070 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":796,"thread_start_us":448,"threads_started":1}
I20260806 22:33:52.896580 20375 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7baa9daa22a4bee90b1af94d37ba405" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33241 } }
I20260806 22:33:52.897082 20375 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:52.897221 20375 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7baa9daa22a4bee90b1af94d37ba405, State: Initialized, Role: FOLLOWER
I20260806 22:33:52.897646 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:33:52.897996 20375 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7baa9daa22a4bee90b1af94d37ba405" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33241 } }
I20260806 22:33:52.898281 20375 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:33:52.898449 20375 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:33:52.898643 20375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:52.901309 20375 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7baa9daa22a4bee90b1af94d37ba405" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33241 } }
I20260806 22:33:52.901989 20375 leader_election.cc:302] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7baa9daa22a4bee90b1af94d37ba405; no voters:
I20260806 22:33:52.902853 20375 leader_election.cc:288] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:33:52.903053 20384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:33:52.904449 20384 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [term 1 LEADER]: Becoming Leader. State: Replica: c7baa9daa22a4bee90b1af94d37ba405, State: Running, Role: LEADER
I20260806 22:33:52.905269 20384 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7baa9daa22a4bee90b1af94d37ba405" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33241 } }
I20260806 22:33:52.906042 20375 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:33:52.908872 20387 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7baa9daa22a4bee90b1af94d37ba405. Latest consensus state: current_term: 1 leader_uuid: "c7baa9daa22a4bee90b1af94d37ba405" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7baa9daa22a4bee90b1af94d37ba405" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33241 } } }
I20260806 22:33:52.909402 20387 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [sys.catalog]: This master's current role is: LEADER
I20260806 22:33:52.909332 20386 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7baa9daa22a4bee90b1af94d37ba405" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7baa9daa22a4bee90b1af94d37ba405" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33241 } } }
I20260806 22:33:52.909756 20386 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405 [sys.catalog]: This master's current role is: LEADER
I20260806 22:33:52.910719 20395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:33:52.923873 20395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:33:52.945339 20395 catalog_manager.cc:1347] Generated new cluster ID: 48bf24c397f3445eb8ec5dbcdc7aaa20
I20260806 22:33:52.945515 20395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:33:52.963513 20395 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:33:52.965524 20395 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:33:52.976262 20395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7baa9daa22a4bee90b1af94d37ba405: Generated new TSK 0
I20260806 22:33:52.977650 20395 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:33:52.982156 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:52.967894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49002 (local address 127.15.198.190:33241)
0806 22:33:52.968140 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:52.968148 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:52.971837 (+ 3689us) server_negotiation.cc:407] Connection header received
0806 22:33:52.972110 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:52.972117 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:52.972225 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:52.972367 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:52.974077 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:52.975217 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:52.977382 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:52.978164 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:52.978338 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:52.979964 (+ 1626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:52.979969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:52.979977 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:33:52.981165 (+ 1188us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:52.981258 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:33:52.981291 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:52.981407 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:52.981620 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 22:33:52.981740 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 22:33:52.993155 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 20280
I20260806 22:33:52.994894 20369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:53.100678 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 20280
2026-08-06T22:33:53Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2464 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:33:53 dist-test-slave-h3ft krb5kdc[20424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:33:53 dist-test-slave-h3ft krb5kdc[20424](info): set up 2 sockets
Aug 06 22:33:53 dist-test-slave-h3ft krb5kdc[20424](info): commencing operation
krb5kdc: starting...
W20260806 22:33:54.499549 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.344s user 0.005s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:33:54 dist-test-slave-h3ft krb5kdc[20424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:33:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:33:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:33:54.820216 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:46331
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:38591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:46331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:55.290253 20446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:55.290786 20446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:55.291004 20446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:55.307376 20446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:33:55.307644 20446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:55.307755 20446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:55.307873 20446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:33:55.307973 20446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:33:55.326395 20446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:38591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:46331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:46331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:55.329025 20446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:55.331499 20446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:55.348881 20452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:55.348973 20456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:55.348940 20453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:55.349907 20446 server_base.cc:1034] running on GCE node
I20260806 22:33:55.351263 20446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:55.353853 20446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:55.355271 20446 hybrid_clock.cc:648] HybridClock initialized: now 1786055635355201 us; error 60 us; skew 500 ppm
I20260806 22:33:55.355821 20446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:55 dist-test-slave-h3ft krb5kdc[20424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055635, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:55.360345 20446 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:33:55.362655 20446 webserver.cc:460] Webserver started at http://127.15.198.190:37927/ using document root <none> and password file <none>
I20260806 22:33:55.363783 20446 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:55.363971 20446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:55.364391 20446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:55.368764 20446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "c720e8b41456493d83523d84b8746c1f"
format_stamp: "Formatted at 2026-08-06 22:33:55 on dist-test-slave-h3ft"
server_key: "b05bc2d94a0c1a9dd8293e67d6e2a6c8"
server_key_iv: "4d977ac33ea932f486d6ed4284e0bd02"
server_key_version: "encryptionkey@0"
I20260806 22:33:55.369850 20446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "c720e8b41456493d83523d84b8746c1f"
format_stamp: "Formatted at 2026-08-06 22:33:55 on dist-test-slave-h3ft"
server_key: "b05bc2d94a0c1a9dd8293e67d6e2a6c8"
server_key_iv: "4d977ac33ea932f486d6ed4284e0bd02"
server_key_version: "encryptionkey@0"
I20260806 22:33:55.377192 20446 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.004s
I20260806 22:33:55.382206 20465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:55.384358 20446 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:33:55.384630 20446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "c720e8b41456493d83523d84b8746c1f"
format_stamp: "Formatted at 2026-08-06 22:33:55 on dist-test-slave-h3ft"
server_key: "b05bc2d94a0c1a9dd8293e67d6e2a6c8"
server_key_iv: "4d977ac33ea932f486d6ed4284e0bd02"
server_key_version: "encryptionkey@0"
I20260806 22:33:55.384914 20446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:55.402586 20446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:55.408836 20446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:55.409250 20446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:55.432866 20446 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:46331
I20260806 22:33:55.432981 20539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:46331 every 8 connection(s)
I20260806 22:33:55.434794 20446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:33:55.437114 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 20446
I20260806 22:33:55.437390 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:33:55.438120 16154 external_mini_cluster.cc:1428] Setting key 9a71e8f3602630b7f203144dfcc88ce2
I20260806 22:33:55.440675 20543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 22:33:55 dist-test-slave-h3ft krb5kdc[20424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:55.459151 20543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f: Bootstrap starting.
I20260806 22:33:55.464301 20543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f: Neither blocks nor log segments found. Creating new log.
I20260806 22:33:55.466033 20543 log.cc:824] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:33:55.468418 20546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:55.440655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58336 (local address 127.15.198.190:46331)
0806 22:33:55.441653 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:55.441667 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:55.441721 (+ 54us) server_negotiation.cc:407] Connection header received
0806 22:33:55.443990 (+ 2269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:55.444023 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:55.444586 (+ 563us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:55.445037 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:55.447310 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:55.448864 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:55.450805 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:55.451810 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:55.451979 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:55.458764 (+ 6785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:55.458776 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:55.458801 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:55.458863 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:55.462715 (+ 3852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:55.463923 (+ 1208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:55.463933 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:55.463941 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:55.464161 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:55.464720 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:55.464724 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:55.464726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:55.465471 (+ 745us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:55.465771 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:55.466443 (+ 672us) server_negotiation.cc:299] Negotiation successful
0806 22:33:55.467092 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":439,"threads_started":1}
I20260806 22:33:55.470088 20543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f: No bootstrap required, opened a new log
I20260806 22:33:55.480113 20543 raft_consensus.cc:348] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c720e8b41456493d83523d84b8746c1f" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46331 } }
I20260806 22:33:55.480916 20543 raft_consensus.cc:374] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:33:55.481071 20543 raft_consensus.cc:729] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c720e8b41456493d83523d84b8746c1f, State: Initialized, Role: FOLLOWER
I20260806 22:33:55.481822 20543 consensus_queue.cc:260] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c720e8b41456493d83523d84b8746c1f" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46331 } }
I20260806 22:33:55.482095 20543 raft_consensus.cc:388] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:33:55.482211 20543 raft_consensus.cc:482] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:33:55.482373 20543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:33:55.485185 20543 raft_consensus.cc:504] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c720e8b41456493d83523d84b8746c1f" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46331 } }
I20260806 22:33:55.486251 20543 leader_election.cc:302] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c720e8b41456493d83523d84b8746c1f; no voters:
I20260806 22:33:55.487223 20543 leader_election.cc:288] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:33:55.487439 20551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:33:55.488276 20551 raft_consensus.cc:686] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [term 1 LEADER]: Becoming Leader. State: Replica: c720e8b41456493d83523d84b8746c1f, State: Running, Role: LEADER
I20260806 22:33:55.488955 20551 consensus_queue.cc:237] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c720e8b41456493d83523d84b8746c1f" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46331 } }
I20260806 22:33:55.490109 20543 sys_catalog.cc:564] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:33:55.492672 20552 sys_catalog.cc:455] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c720e8b41456493d83523d84b8746c1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c720e8b41456493d83523d84b8746c1f" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46331 } } }
I20260806 22:33:55.493880 20552 sys_catalog.cc:458] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [sys.catalog]: This master's current role is: LEADER
I20260806 22:33:55.493345 20553 sys_catalog.cc:455] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c720e8b41456493d83523d84b8746c1f. Latest consensus state: current_term: 1 leader_uuid: "c720e8b41456493d83523d84b8746c1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c720e8b41456493d83523d84b8746c1f" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46331 } } }
I20260806 22:33:55.494127 20553 sys_catalog.cc:458] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f [sys.catalog]: This master's current role is: LEADER
I20260806 22:33:55.496917 20556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:33:55.504289 20556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:33:55.518663 20556 catalog_manager.cc:1347] Generated new cluster ID: 99dc23badbf646729c67152da50130a3
I20260806 22:33:55.518836 20556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:33:55.547436 20556 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:33:55.549156 20556 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:33:55.558079 20556 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c720e8b41456493d83523d84b8746c1f: Generated new TSK 0
I20260806 22:33:55.559518 20556 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
I20260806 22:33:55.661158 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:46331
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:38591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:56.143445 20591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:56.143957 20591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:56.144152 20591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:56.152902 20591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:56.153152 20591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:56.153375 20591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:33:56.165045 20591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:38591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:46331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:56.167080 20591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:56.168964 20591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:56.186116 20598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:56.186030 20601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:56.186071 20599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:56.187127 20591 server_base.cc:1034] running on GCE node
I20260806 22:33:56.188043 20591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:56.189380 20591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:56.190610 20591 hybrid_clock.cc:648] HybridClock initialized: now 1786055636190545 us; error 64 us; skew 500 ppm
I20260806 22:33:56.191095 20591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:56 dist-test-slave-h3ft krb5kdc[20424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055636, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:56.195461 20591 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:33:56.197584 20591 webserver.cc:460] Webserver started at http://127.15.198.129:41897/ using document root <none> and password file <none>
I20260806 22:33:56.198632 20591 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:56.198820 20591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:56.199272 20591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:56.203097 20591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "b860f3f5b9f449a380e6c62a4aecf13e"
format_stamp: "Formatted at 2026-08-06 22:33:56 on dist-test-slave-h3ft"
server_key: "b84f3b170fa44eb5f3ef2f99335c0f44"
server_key_iv: "ee66d0a53a73012eec7b5dd4f97ba60f"
server_key_version: "encryptionkey@0"
I20260806 22:33:56.204051 20591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "b860f3f5b9f449a380e6c62a4aecf13e"
format_stamp: "Formatted at 2026-08-06 22:33:56 on dist-test-slave-h3ft"
server_key: "b84f3b170fa44eb5f3ef2f99335c0f44"
server_key_iv: "ee66d0a53a73012eec7b5dd4f97ba60f"
server_key_version: "encryptionkey@0"
I20260806 22:33:56.210829 20591 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 22:33:56.215564 20610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:56.217476 20591 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 22:33:56.217772 20591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "b860f3f5b9f449a380e6c62a4aecf13e"
format_stamp: "Formatted at 2026-08-06 22:33:56 on dist-test-slave-h3ft"
server_key: "b84f3b170fa44eb5f3ef2f99335c0f44"
server_key_iv: "ee66d0a53a73012eec7b5dd4f97ba60f"
server_key_version: "encryptionkey@0"
I20260806 22:33:56.218044 20591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:56.245306 20591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:56.251727 20591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:56.252177 20591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:56.253638 20591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:33:56.255959 20591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:56.256106 20591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:56.256300 20591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:56.256383 20591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:56.298697 20591 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:37679
I20260806 22:33:56.298758 20760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:37679 every 8 connection(s)
I20260806 22:33:56.300599 20591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:33:56.301928 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 20591
I20260806 22:33:56.302244 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:33:56.303256 16154 external_mini_cluster.cc:1428] Setting key 9265113d258e649fd9c505b31976256e
Aug 06 22:33:56 dist-test-slave-h3ft krb5kdc[20424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055636, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:56.330248 20763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:56.304540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:45371 (local address 127.15.198.190:46331)
0806 22:33:56.305173 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:56.305180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:56.306812 (+ 1632us) server_negotiation.cc:407] Connection header received
0806 22:33:56.309294 (+ 2482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:56.309303 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:56.309431 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:56.309578 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:56.311830 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:56.312712 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:56.314634 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:56.315225 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:56.315387 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:56.320896 (+ 5509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:56.320908 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:56.320917 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:56.320990 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:56.325372 (+ 4382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:56.326584 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:56.326594 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:56.326600 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:56.326799 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:56.327559 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:56.327567 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:56.327573 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:56.328091 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:56.328301 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:56.329318 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0806 22:33:56.329683 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":331,"threads_started":1}
I20260806 22:33:56.331490 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:56.305489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:46331 (local address 127.15.198.129:45371)
0806 22:33:56.306608 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:56.306660 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:56.308887 (+ 2227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:56.309920 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:56.309932 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:56.310658 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:56.311574 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:56.311604 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:56.312939 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:56.312944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:56.314441 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:56.314456 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:56.315569 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:56.315579 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:56.315867 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:56.316916 (+ 1049us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:56.316962 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:56.320508 (+ 3546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:56.325621 (+ 5113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:56.325632 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:56.325656 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:56.326377 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:56.327066 (+ 689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:56.327080 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:56.327083 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:56.327398 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:56.328271 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:56.328279 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:56.328774 (+ 495us) client_negotiation.cc:769] Sending connection context
0806 22:33:56.329140 (+ 366us) client_negotiation.cc:240] Negotiation successful
0806 22:33:56.329848 (+ 708us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":457,"threads_started":1}
I20260806 22:33:56.333859 20761 heartbeater.cc:347] Connected to a master server at 127.15.198.190:46331
I20260806 22:33:56.334342 20761 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
I20260806 22:33:56.335542 20761 heartbeater.cc:511] Master 127.15.198.190:46331 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
I20260806 22:33:56.338896 20490 ts_manager.cc:194] Registered new tserver with Master: b860f3f5b9f449a380e6c62a4aecf13e (127.15.198.129:37679)
I20260806 22:33:56.340845 20490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:45371
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
I20260806 22:33:56.376157 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:46331
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:38591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:33:56.810627 20771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:56.811278 20771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:56.811579 20771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:56.820422 20771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:56.820595 20771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:56.820813 20771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:33:56.833457 20771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:38591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:46331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:56.835541 20771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:56.837389 20771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:56.857604 20779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:56.857511 20783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:56.857580 20780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:56.858680 20771 server_base.cc:1034] running on GCE node
I20260806 22:33:56.859620 20771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:56.860903 20771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:56.862145 20771 hybrid_clock.cc:648] HybridClock initialized: now 1786055636862072 us; error 57 us; skew 500 ppm
I20260806 22:33:56.862586 20771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:56 dist-test-slave-h3ft krb5kdc[20424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055636, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:56.866866 20771 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:33:56.868860 20771 webserver.cc:460] Webserver started at http://127.15.198.130:45409/ using document root <none> and password file <none>
I20260806 22:33:56.870115 20771 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:56.870257 20771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:56.870608 20771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:56.874622 20771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "42cdba2b15e14c08b16358159049c8ee"
format_stamp: "Formatted at 2026-08-06 22:33:56 on dist-test-slave-h3ft"
server_key: "eaccded025de5a4b6a8dbe42de283f9a"
server_key_iv: "33eafb9773879175d62d1b3f6967744e"
server_key_version: "encryptionkey@0"
I20260806 22:33:56.875710 20771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "42cdba2b15e14c08b16358159049c8ee"
format_stamp: "Formatted at 2026-08-06 22:33:56 on dist-test-slave-h3ft"
server_key: "eaccded025de5a4b6a8dbe42de283f9a"
server_key_iv: "33eafb9773879175d62d1b3f6967744e"
server_key_version: "encryptionkey@0"
I20260806 22:33:56.883232 20771 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 22:33:56.888324 20792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:56.890640 20771 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260806 22:33:56.890910 20771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "42cdba2b15e14c08b16358159049c8ee"
format_stamp: "Formatted at 2026-08-06 22:33:56 on dist-test-slave-h3ft"
server_key: "eaccded025de5a4b6a8dbe42de283f9a"
server_key_iv: "33eafb9773879175d62d1b3f6967744e"
server_key_version: "encryptionkey@0"
I20260806 22:33:56.891273 20771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:56.910576 20771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:56.916554 20771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:56.916960 20771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:56.918334 20771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:33:56.920363 20771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:56.920462 20771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:56.920647 20771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:56.920715 20771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:56.962778 20771 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:41487
I20260806 22:33:56.962817 20944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:41487 every 8 connection(s)
I20260806 22:33:56.964601 20771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:33:56.971504 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 20771
I20260806 22:33:56.971766 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:33:56.972302 16154 external_mini_cluster.cc:1428] Setting key c0e6f4fa0ff4706140a79468f40215b0
Aug 06 22:33:56 dist-test-slave-h3ft krb5kdc[20424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055636, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:56.991780 20948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:56.968139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:41609 (local address 127.15.198.190:46331)
0806 22:33:56.968756 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:56.968762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:56.969966 (+ 1204us) server_negotiation.cc:407] Connection header received
0806 22:33:56.972349 (+ 2383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:56.972356 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:56.972472 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:56.972645 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:56.974898 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:56.975811 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:56.977652 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:56.978180 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:56.978334 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:56.983362 (+ 5028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:56.983372 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:56.983380 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:56.983448 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:56.987114 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:56.988315 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:56.988321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:56.988323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:56.988506 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:56.989189 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:56.989197 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:56.989201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:56.989576 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:56.989757 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:56.990993 (+ 1236us) server_negotiation.cc:299] Negotiation successful
0806 22:33:56.991377 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":349,"threads_started":1}
I20260806 22:33:56.993692 20950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:56.968690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:46331 (local address 127.15.198.130:41609)
0806 22:33:56.969675 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:56.969770 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:56.972012 (+ 2242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:56.972962 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:56.972973 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:56.973741 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:56.974698 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:56.974723 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:56.976028 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:56.976032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:56.977473 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:56.977488 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:56.978460 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:56.978468 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:56.978772 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:56.979735 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:56.979765 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:56.982934 (+ 3169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:56.987315 (+ 4381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:56.987330 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:56.987359 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:56.988152 (+ 793us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:56.988643 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:56.988651 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:56.988658 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:56.988970 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:56.989734 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:56.989747 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:56.990420 (+ 673us) client_negotiation.cc:769] Sending connection context
0806 22:33:56.990829 (+ 409us) client_negotiation.cc:240] Negotiation successful
0806 22:33:56.991651 (+ 822us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":371,"threads_started":1}
I20260806 22:33:56.995822 20945 heartbeater.cc:347] Connected to a master server at 127.15.198.190:46331
I20260806 22:33:56.996335 20945 heartbeater.cc:464] Registering TS with master...
I20260806 22:33:56.997555 20945 heartbeater.cc:511] Master 127.15.198.190:46331 requested a full tablet report, sending...
I20260806 22:33:56.999784 20490 ts_manager.cc:194] Registered new tserver with Master: 42cdba2b15e14c08b16358159049c8ee (127.15.198.130:41487)
I20260806 22:33:57.000942 20490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:41609
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
I20260806 22:33:57.045550 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:46331
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:38591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:33:57.346282 20761 heartbeater.cc:503] Master 127.15.198.190:46331 was elected leader, sending a full tablet report...
W20260806 22:33:57.491307 20953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:33:57.491742 20953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:33:57.491902 20953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:33:57.500877 20953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:33:57.501072 20953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:33:57.501214 20953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:33:57.513146 20953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:38591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:46331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:33:57.515038 20953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:33:57.516785 20953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:33:57.535050 20962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:57.534956 20965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:33:57.535095 20961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:33:57.536177 20953 server_base.cc:1034] running on GCE node
I20260806 22:33:57.537019 20953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:33:57.538329 20953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:33:57.539764 20953 hybrid_clock.cc:648] HybridClock initialized: now 1786055637539643 us; error 83 us; skew 500 ppm
I20260806 22:33:57.540299 20953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:33:57 dist-test-slave-h3ft krb5kdc[20424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055637, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:33:57.544711 20953 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:33:57.546700 20953 webserver.cc:460] Webserver started at http://127.15.198.131:42805/ using document root <none> and password file <none>
I20260806 22:33:57.547755 20953 fs_manager.cc:359] Metadata directory not provided
I20260806 22:33:57.547940 20953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:33:57.548344 20953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:33:57.552469 20953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "5ab6f8f13ae74e9e806ae9753aa7c29e"
format_stamp: "Formatted at 2026-08-06 22:33:57 on dist-test-slave-h3ft"
server_key: "7824da6ab71c152c5b223bcd3c1d35f6"
server_key_iv: "549716cc8cafe12972c535176613b81c"
server_key_version: "encryptionkey@0"
I20260806 22:33:57.553665 20953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "5ab6f8f13ae74e9e806ae9753aa7c29e"
format_stamp: "Formatted at 2026-08-06 22:33:57 on dist-test-slave-h3ft"
server_key: "7824da6ab71c152c5b223bcd3c1d35f6"
server_key_iv: "549716cc8cafe12972c535176613b81c"
server_key_version: "encryptionkey@0"
I20260806 22:33:57.561197 20953 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260806 22:33:57.566249 20976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:57.568332 20953 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:33:57.568619 20953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "5ab6f8f13ae74e9e806ae9753aa7c29e"
format_stamp: "Formatted at 2026-08-06 22:33:57 on dist-test-slave-h3ft"
server_key: "7824da6ab71c152c5b223bcd3c1d35f6"
server_key_iv: "549716cc8cafe12972c535176613b81c"
server_key_version: "encryptionkey@0"
I20260806 22:33:57.568908 20953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:33:57.595710 20953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:33:57.602694 20953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:33:57.603192 20953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:33:57.604648 20953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:33:57.606546 20953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:33:57.606647 20953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:57.606856 20953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:33:57.606920 20953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:33:57.654567 20953 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:43277
I20260806 22:33:57.654615 21129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:43277 every 8 connection(s)
I20260806 22:33:57.657085 20953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:33:57.662251 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 20953
I20260806 22:33:57.662585 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:33:57.663218 16154 external_mini_cluster.cc:1428] Setting key 520ef0409d363f06710811e716371fdc
Aug 06 22:33:57 dist-test-slave-h3ft krb5kdc[20424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055637, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:57.686863 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:57.661592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:48555 (local address 127.15.198.190:46331)
0806 22:33:57.662299 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:57.662309 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:57.663850 (+ 1541us) server_negotiation.cc:407] Connection header received
0806 22:33:57.666467 (+ 2617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:57.666472 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:57.666601 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:57.666705 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:57.668930 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:57.669769 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:57.672423 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:57.672945 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:57.673074 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:57.678450 (+ 5376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:57.678464 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:57.678468 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:57.678529 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:57.682311 (+ 3782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:57.683622 (+ 1311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:57.683628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:57.683630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:57.683798 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:57.684413 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:57.684417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:57.684420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:57.684826 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:57.684972 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:57.686217 (+ 1245us) server_negotiation.cc:299] Negotiation successful
0806 22:33:57.686514 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":327,"threads_started":1}
I20260806 22:33:57.688272 21133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:57.662380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:46331 (local address 127.15.198.131:48555)
0806 22:33:57.663582 (+ 1202us) negotiation.cc:107] Waiting for socket to connect
0806 22:33:57.663657 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 22:33:57.666084 (+ 2427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:33:57.667122 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:33:57.667134 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:33:57.667867 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:33:57.668718 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:57.668741 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:57.669984 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:57.669988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:57.672238 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:33:57.672255 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:33:57.673262 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:33:57.673268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:33:57.673565 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:57.674627 (+ 1062us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:33:57.674661 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:33:57.678002 (+ 3341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:33:57.682558 (+ 4556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:57.682569 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:57.682593 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:57.683450 (+ 857us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:57.683971 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:33:57.683976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:33:57.683978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:33:57.684272 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:33:57.685110 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:33:57.685135 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:33:57.685705 (+ 570us) client_negotiation.cc:769] Sending connection context
0806 22:33:57.686052 (+ 347us) client_negotiation.cc:240] Negotiation successful
0806 22:33:57.686656 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"thread_start_us":512,"threads_started":1}
I20260806 22:33:57.690435 21130 heartbeater.cc:347] Connected to a master server at 127.15.198.190:46331
I20260806 22:33:57.690905 21130 heartbeater.cc:464] Registering TS with master...
I20260806 22:33:57.692288 21130 heartbeater.cc:511] Master 127.15.198.190:46331 requested a full tablet report, sending...
I20260806 22:33:57.695576 20490 ts_manager.cc:194] Registered new tserver with Master: 5ab6f8f13ae74e9e806ae9753aa7c29e (127.15.198.131:43277)
I20260806 22:33:57.697391 20490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:48555
I20260806 22:33:57.709298 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:33:57 dist-test-slave-h3ft krb5kdc[20424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055637, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 22:33:57 dist-test-slave-h3ft krb5kdc[20424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055637, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:33:57.792729 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:57.772362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58348 (local address 127.15.198.190:46331)
0806 22:33:57.772666 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:57.772671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:57.773361 (+ 690us) server_negotiation.cc:407] Connection header received
0806 22:33:57.773629 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:57.773634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:57.773741 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:57.773854 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:57.775886 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:57.776802 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:57.778369 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:57.778868 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:57.779081 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:57.784590 (+ 5509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:57.784599 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:57.784603 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:57.784665 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:57.788235 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:57.789387 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:57.789394 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:57.789397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:57.789633 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:57.790310 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:57.790316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:57.790319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:57.791286 (+ 967us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:57.791554 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:57.792134 (+ 580us) server_negotiation.cc:299] Negotiation successful
0806 22:33:57.792404 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
W20260806 22:33:57.793591 20490 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58348
I20260806 22:33:57.796543 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 20591
Aug 06 22:33:57 dist-test-slave-h3ft krb5kdc[20424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055637, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:33:57.819458 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:57.798332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45178 (local address 127.15.198.129:37679)
0806 22:33:57.799273 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:57.799282 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:57.799318 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:33:57.799413 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:57.799422 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:57.799752 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:57.799936 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:57.801717 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:57.803105 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:57.804769 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:57.805525 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:57.805661 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:57.811279 (+ 5618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:57.811302 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:57.811334 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:57.811405 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:57.815351 (+ 3946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:57.816442 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:57.816453 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:57.816456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:57.816662 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:57.817206 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:57.817210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:57.817213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:57.817763 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:57.818120 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:57.818597 (+ 477us) server_negotiation.cc:299] Negotiation successful
0806 22:33:57.818876 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":475,"threads_started":1}
W20260806 22:33:57.821354 20752 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45178
W20260806 22:33:57.823052 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 20591: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:33:57.823345 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 20591
I20260806 22:33:57.862687 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 20771
Aug 06 22:33:57 dist-test-slave-h3ft krb5kdc[20424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055637, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.198.130@KRBTEST.COM
I20260806 22:33:57.885192 21144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:57.864086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33088 (local address 127.15.198.130:41487)
0806 22:33:57.864882 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:57.864891 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:57.864926 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:33:57.865027 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:57.865036 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:57.865341 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:57.865588 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:57.867277 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:57.868489 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:57.870014 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:57.870780 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:57.870968 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:57.876630 (+ 5662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:57.876651 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:57.876673 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:57.876729 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:57.880960 (+ 4231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:57.882062 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:57.882072 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:57.882075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:57.882268 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:57.882835 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:57.882840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:57.882843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:57.883655 (+ 812us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:57.883979 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:57.884463 (+ 484us) server_negotiation.cc:299] Negotiation successful
0806 22:33:57.884773 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":413,"threads_started":1}
W20260806 22:33:57.887192 20940 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33088
W20260806 22:33:57.888797 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 20771: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:33:57.889036 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 20771
I20260806 22:33:57.919067 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 20953
Aug 06 22:33:57 dist-test-slave-h3ft krb5kdc[20424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055637, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.198.131@KRBTEST.COM
I20260806 22:33:57.942355 21145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:33:57.920538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48748 (local address 127.15.198.131:43277)
0806 22:33:57.921414 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0806 22:33:57.921424 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:33:57.921463 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:33:57.921548 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:33:57.921556 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:33:57.921850 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:33:57.922020 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:33:57.923950 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:57.925272 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:57.926847 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:33:57.927793 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:33:57.927926 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:33:57.933876 (+ 5950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:33:57.933899 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:33:57.933924 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:33:57.933992 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:33:57.938299 (+ 4307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:57.939537 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:57.939547 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:57.939550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:57.939728 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:33:57.940234 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:33:57.940238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:33:57.940241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:33:57.940784 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:33:57.941097 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0806 22:33:57.941533 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 22:33:57.941863 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":422,"threads_started":1}
W20260806 22:33:57.944279 21124 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48748
W20260806 22:33:57.945843 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 20953: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:33:57.946070 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 20953
I20260806 22:33:57.974581 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 20446
I20260806 22:33:57.976249 20534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:33:58.101491 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 20446
2026-08-06T22:33:58Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (5008 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:33:58 dist-test-slave-h3ft krb5kdc[21149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:33:58 dist-test-slave-h3ft krb5kdc[21149](info): set up 2 sockets
Aug 06 22:33:58 dist-test-slave-h3ft krb5kdc[21149](info): commencing operation
krb5kdc: starting...
W20260806 22:33:59.469802 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.003s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:33:59 dist-test-slave-h3ft krb5kdc[21149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:33:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:33:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:33:59.781137 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:43597
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:45693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:43597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:00.226719 21164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:00.227315 21164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:00.227525 21164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:00.237180 21164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:34:00.237406 21164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:00.237535 21164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:00.237668 21164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:34:00.237747 21164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:34:00.250876 21164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:45693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:43597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:43597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:00.253134 21164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:00.255290 21164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:00.271800 21176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:00.271700 21173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:00.271700 21174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:00.272657 21164 server_base.cc:1034] running on GCE node
I20260806 22:34:00.273787 21164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:00.275892 21164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:00.277138 21164 hybrid_clock.cc:648] HybridClock initialized: now 1786055640277086 us; error 40 us; skew 500 ppm
I20260806 22:34:00.277621 21164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:00 dist-test-slave-h3ft krb5kdc[21149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055640, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:00.287948 21164 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:34:00.290330 21164 webserver.cc:460] Webserver started at http://127.15.198.190:33463/ using document root <none> and password file <none>
I20260806 22:34:00.291627 21164 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:00.291815 21164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:00.292258 21164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:00.296556 21164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "47e39f0ab4ca4bce8401ccd44a39baf5"
format_stamp: "Formatted at 2026-08-06 22:34:00 on dist-test-slave-h3ft"
server_key: "08cffc232a91e4703c90b9f1559950f9"
server_key_iv: "96afa8725ed0a8b3f74e165d785283aa"
server_key_version: "encryptionkey@0"
I20260806 22:34:00.297617 21164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "47e39f0ab4ca4bce8401ccd44a39baf5"
format_stamp: "Formatted at 2026-08-06 22:34:00 on dist-test-slave-h3ft"
server_key: "08cffc232a91e4703c90b9f1559950f9"
server_key_iv: "96afa8725ed0a8b3f74e165d785283aa"
server_key_version: "encryptionkey@0"
I20260806 22:34:00.304904 21164 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 22:34:00.310293 21186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:00.312523 21164 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 22:34:00.312820 21164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "47e39f0ab4ca4bce8401ccd44a39baf5"
format_stamp: "Formatted at 2026-08-06 22:34:00 on dist-test-slave-h3ft"
server_key: "08cffc232a91e4703c90b9f1559950f9"
server_key_iv: "96afa8725ed0a8b3f74e165d785283aa"
server_key_version: "encryptionkey@0"
I20260806 22:34:00.313088 21164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:00.341619 21164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:00.347749 21164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:00.348196 21164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:00.373642 21164 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:43597
I20260806 22:34:00.373680 21263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:43597 every 8 connection(s)
I20260806 22:34:00.375738 21164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:34:00.377414 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 21164
I20260806 22:34:00.377738 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:34:00.378531 16154 external_mini_cluster.cc:1428] Setting key 22e5d60900bbce5a16ba93db7fb37ad3
I20260806 22:34:00.382336 21264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 22:34:00 dist-test-slave-h3ft krb5kdc[21149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:34:00.399168 21264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5: Bootstrap starting.
I20260806 22:34:00.405041 21264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:00.406723 21264 log.cc:824] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:00.411325 21264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5: No bootstrap required, opened a new log
I20260806 22:34:00.411482 21267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:00.381508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38254 (local address 127.15.198.190:43597)
0806 22:34:00.382695 (+ 1187us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:00.382716 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:00.382803 (+ 87us) server_negotiation.cc:407] Connection header received
0806 22:34:00.385639 (+ 2836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:00.385678 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:00.386351 (+ 673us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:00.386985 (+ 634us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:00.389056 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:00.391492 (+ 2436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:00.393121 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:00.394338 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:00.394572 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:00.400209 (+ 5637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:00.400224 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:00.400252 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:00.400315 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:00.405326 (+ 5011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:00.406414 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:00.406428 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:00.406457 (+ 29us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:00.406770 (+ 313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:00.407431 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:00.407437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:00.407443 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:00.408324 (+ 881us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:00.408702 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:00.409189 (+ 487us) server_negotiation.cc:299] Negotiation successful
0806 22:34:00.409764 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":862,"thread_start_us":616,"threads_started":1}
I20260806 22:34:00.420753 21264 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e39f0ab4ca4bce8401ccd44a39baf5" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 43597 } }
I20260806 22:34:00.421383 21264 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:00.421526 21264 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47e39f0ab4ca4bce8401ccd44a39baf5, State: Initialized, Role: FOLLOWER
I20260806 22:34:00.422322 21264 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e39f0ab4ca4bce8401ccd44a39baf5" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 43597 } }
I20260806 22:34:00.422616 21264 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:34:00.422778 21264 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:34:00.423035 21264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:00.425666 21264 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e39f0ab4ca4bce8401ccd44a39baf5" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 43597 } }
I20260806 22:34:00.426447 21264 leader_election.cc:302] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47e39f0ab4ca4bce8401ccd44a39baf5; no voters:
I20260806 22:34:00.427384 21264 leader_election.cc:288] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:34:00.427577 21273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:00.428390 21273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [term 1 LEADER]: Becoming Leader. State: Replica: 47e39f0ab4ca4bce8401ccd44a39baf5, State: Running, Role: LEADER
I20260806 22:34:00.429242 21273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e39f0ab4ca4bce8401ccd44a39baf5" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 43597 } }
I20260806 22:34:00.430413 21264 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:34:00.433452 21276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47e39f0ab4ca4bce8401ccd44a39baf5. Latest consensus state: current_term: 1 leader_uuid: "47e39f0ab4ca4bce8401ccd44a39baf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e39f0ab4ca4bce8401ccd44a39baf5" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 43597 } } }
I20260806 22:34:00.434630 21276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:00.433944 21275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47e39f0ab4ca4bce8401ccd44a39baf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e39f0ab4ca4bce8401ccd44a39baf5" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 43597 } } }
I20260806 22:34:00.435213 21275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:00.439024 21283 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:34:00.446244 21283 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:34:00.460642 21283 catalog_manager.cc:1347] Generated new cluster ID: c06f08097fda48c7b0cf6db88aad2331
I20260806 22:34:00.460896 21283 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:34:00.474498 21283 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:34:00.476306 21283 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:34:00.489130 21283 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47e39f0ab4ca4bce8401ccd44a39baf5: Generated new TSK 0
I20260806 22:34:00.490449 21283 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
I20260806 22:34:00.573673 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:43597
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:45693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:01.027554 21302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:01.027945 21302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:01.028126 21302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:01.036619 21302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:01.036791 21302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:01.036998 21302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:34:01.048877 21302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:45693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:43597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:01.050832 21302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:01.052712 21302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:01.069658 21307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:01.069965 21311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:01.069697 21309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:01.071398 21302 server_base.cc:1034] running on GCE node
I20260806 22:34:01.072333 21302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:01.073755 21302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:01.075109 21302 hybrid_clock.cc:648] HybridClock initialized: now 1786055641074965 us; error 53 us; skew 500 ppm
I20260806 22:34:01.075570 21302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:01 dist-test-slave-h3ft krb5kdc[21149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055641, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:01.080348 21302 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:34:01.082610 21302 webserver.cc:460] Webserver started at http://127.15.198.129:37497/ using document root <none> and password file <none>
I20260806 22:34:01.083703 21302 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:01.083884 21302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:01.084335 21302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:01.088719 21302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "ae8e4bc7d6f9427092123eb46959443f"
format_stamp: "Formatted at 2026-08-06 22:34:01 on dist-test-slave-h3ft"
server_key: "d55051dc45df6679848378a99cb4a406"
server_key_iv: "07da333d9c7d5cf208274c6fef18d550"
server_key_version: "encryptionkey@0"
I20260806 22:34:01.089812 21302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "ae8e4bc7d6f9427092123eb46959443f"
format_stamp: "Formatted at 2026-08-06 22:34:01 on dist-test-slave-h3ft"
server_key: "d55051dc45df6679848378a99cb4a406"
server_key_iv: "07da333d9c7d5cf208274c6fef18d550"
server_key_version: "encryptionkey@0"
I20260806 22:34:01.097530 21302 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 22:34:01.103215 21318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:01.105515 21302 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 22:34:01.105806 21302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "ae8e4bc7d6f9427092123eb46959443f"
format_stamp: "Formatted at 2026-08-06 22:34:01 on dist-test-slave-h3ft"
server_key: "d55051dc45df6679848378a99cb4a406"
server_key_iv: "07da333d9c7d5cf208274c6fef18d550"
server_key_version: "encryptionkey@0"
I20260806 22:34:01.106161 21302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:01.134500 21302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:01.141409 21302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:01.141757 21302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:01.142988 21302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:01.144824 21302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:01.144918 21302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:01.145247 21302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:01.145324 21302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:01.187597 21302 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:33139
I20260806 22:34:01.187634 21480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:33139 every 8 connection(s)
I20260806 22:34:01.189455 21302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:34:01.191349 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 21302
I20260806 22:34:01.191620 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:34:01.192600 16154 external_mini_cluster.cc:1428] Setting key ff7a7bf66ff54c53aea95283b69e8e2c
Aug 06 22:34:01 dist-test-slave-h3ft krb5kdc[21149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055641, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:34:01.221194 21486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:01.193620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:39593 (local address 127.15.198.190:43597)
0806 22:34:01.194255 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:01.194262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:01.195645 (+ 1383us) server_negotiation.cc:407] Connection header received
0806 22:34:01.199194 (+ 3549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:01.199202 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:01.199336 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:01.199462 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:01.202807 (+ 3345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:01.203817 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:01.206990 (+ 3173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:01.207573 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:01.207708 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:01.212694 (+ 4986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:01.212700 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:01.212704 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:01.212754 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:01.216608 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:01.217839 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:01.217846 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:01.217852 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:01.218087 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:01.218691 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:01.218698 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:01.218701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:01.219166 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:01.219363 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:01.220380 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0806 22:34:01.220723 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":340,"threads_started":1}
I20260806 22:34:01.222599 21487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:01.194313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:43597 (local address 127.15.198.129:39593)
0806 22:34:01.195422 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:01.195497 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:01.198585 (+ 3088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:01.199884 (+ 1299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:01.199904 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:01.201058 (+ 1154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:01.202375 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:01.202427 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:01.204102 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:01.204111 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:01.206717 (+ 2606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:01.206750 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:01.207906 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:01.207914 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:01.208173 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:01.209150 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:01.209182 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:01.212302 (+ 3120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:01.216929 (+ 4627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:01.216941 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:01.216967 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:01.217649 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:01.218254 (+ 605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:01.218258 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:01.218261 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:01.218548 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:01.219340 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:01.219351 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:01.219885 (+ 534us) client_negotiation.cc:769] Sending connection context
0806 22:34:01.220243 (+ 358us) client_negotiation.cc:240] Negotiation successful
0806 22:34:01.221020 (+ 777us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":414,"threads_started":1}
I20260806 22:34:01.225282 21482 heartbeater.cc:347] Connected to a master server at 127.15.198.190:43597
I20260806 22:34:01.225762 21482 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
I20260806 22:34:01.226933 21482 heartbeater.cc:511] Master 127.15.198.190:43597 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
I20260806 22:34:01.230854 21212 ts_manager.cc:194] Registered new tserver with Master: ae8e4bc7d6f9427092123eb46959443f (127.15.198.129:33139)
I20260806 22:34:01.233745 21212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:39593
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
I20260806 22:34:01.267829 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:43597
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:45693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:01.727207 21493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:01.728070 21493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:01.728286 21493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:01.739056 21493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:01.739707 21493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:01.740020 21493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:34:01.754858 21493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:45693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:43597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:01.756939 21493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:01.758939 21493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:01.778522 21499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:01.778434 21503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:01.778522 21500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:01.781100 21493 server_base.cc:1034] running on GCE node
I20260806 22:34:01.782203 21493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:01.783851 21493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:01.785207 21493 hybrid_clock.cc:648] HybridClock initialized: now 1786055641785117 us; error 98 us; skew 500 ppm
I20260806 22:34:01.785734 21493 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:01 dist-test-slave-h3ft krb5kdc[21149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055641, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:01.791137 21493 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:34:01.794257 21493 webserver.cc:460] Webserver started at http://127.15.198.130:46021/ using document root <none> and password file <none>
I20260806 22:34:01.795552 21493 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:01.795774 21493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:01.796250 21493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:01.801167 21493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "819cbff6741f4b5d9e37cd5d474c75da"
format_stamp: "Formatted at 2026-08-06 22:34:01 on dist-test-slave-h3ft"
server_key: "d2b36a487bdf61292cb069a7218a94b8"
server_key_iv: "09d143573c143f5c6e18393f24be9cad"
server_key_version: "encryptionkey@0"
I20260806 22:34:01.802615 21493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "819cbff6741f4b5d9e37cd5d474c75da"
format_stamp: "Formatted at 2026-08-06 22:34:01 on dist-test-slave-h3ft"
server_key: "d2b36a487bdf61292cb069a7218a94b8"
server_key_iv: "09d143573c143f5c6e18393f24be9cad"
server_key_version: "encryptionkey@0"
I20260806 22:34:01.811462 21493 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.004s
I20260806 22:34:01.817464 21511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:01.820154 21493 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 22:34:01.820468 21493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "819cbff6741f4b5d9e37cd5d474c75da"
format_stamp: "Formatted at 2026-08-06 22:34:01 on dist-test-slave-h3ft"
server_key: "d2b36a487bdf61292cb069a7218a94b8"
server_key_iv: "09d143573c143f5c6e18393f24be9cad"
server_key_version: "encryptionkey@0"
I20260806 22:34:01.820802 21493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:01.868897 21493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:01.875731 21493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:01.876178 21493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:01.877666 21493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:01.879736 21493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:01.879892 21493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:01.880097 21493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:01.880179 21493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:01.925401 21493 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:46661
I20260806 22:34:01.925494 21664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:46661 every 8 connection(s)
I20260806 22:34:01.927559 21493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:34:01.928131 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 21493
I20260806 22:34:01.928452 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:34:01.929260 16154 external_mini_cluster.cc:1428] Setting key f899406251f54b03069a438d0ba0be92
Aug 06 22:34:01 dist-test-slave-h3ft krb5kdc[21149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055641, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:34:01.959884 21668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:01.931674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:35493 (local address 127.15.198.190:43597)
0806 22:34:01.932375 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:01.932380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:01.933987 (+ 1607us) server_negotiation.cc:407] Connection header received
0806 22:34:01.936830 (+ 2843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:01.936840 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:01.936983 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:01.937140 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:01.940236 (+ 3096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:01.941547 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:01.943878 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:01.944632 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:01.944779 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:01.950888 (+ 6109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:01.950895 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:01.950899 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:01.950953 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:01.954463 (+ 3510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:01.955788 (+ 1325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:01.955793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:01.955796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:01.955984 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:01.956815 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:01.956825 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:01.956832 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:01.957391 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:01.957626 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:01.958823 (+ 1197us) server_negotiation.cc:299] Negotiation successful
0806 22:34:01.959319 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":414,"threads_started":1}
I20260806 22:34:01.960971 21671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:01.932666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:43597 (local address 127.15.198.130:35493)
0806 22:34:01.933750 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:01.933813 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:01.936393 (+ 2580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:01.937582 (+ 1189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:01.937596 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:01.938794 (+ 1198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:01.939870 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:01.939921 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:01.941822 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:01.941827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:01.943623 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:01.943647 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:01.945030 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:01.945040 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:01.945505 (+ 465us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:01.946629 (+ 1124us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:01.946674 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:01.950419 (+ 3745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:01.954768 (+ 4349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:01.954781 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:01.954804 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:01.955578 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:01.956143 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:01.956156 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:01.956159 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:01.956439 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:01.957642 (+ 1203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:01.957653 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:01.958234 (+ 581us) client_negotiation.cc:769] Sending connection context
0806 22:34:01.958601 (+ 367us) client_negotiation.cc:240] Negotiation successful
0806 22:34:01.959330 (+ 729us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":455,"threads_started":1}
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
I20260806 22:34:01.963613 21665 heartbeater.cc:347] Connected to a master server at 127.15.198.190:43597
I20260806 22:34:01.964038 21665 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:01.965109 21665 heartbeater.cc:511] Master 127.15.198.190:43597 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
I20260806 22:34:01.969349 21212 ts_manager.cc:194] Registered new tserver with Master: 819cbff6741f4b5d9e37cd5d474c75da (127.15.198.130:46661)
I20260806 22:34:01.970887 21212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:35493
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
I20260806 22:34:02.012689 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:43597
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:45693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:34:02.239856 21482 heartbeater.cc:503] Master 127.15.198.190:43597 was elected leader, sending a full tablet report...
W20260806 22:34:02.502666 21676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:02.503225 21676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:02.503458 21676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:02.512194 21676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:02.512372 21676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:02.512635 21676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:34:02.524806 21676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:45693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:43597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:02.526822 21676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:02.528887 21676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:02.546764 21686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:02.546684 21687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:02.546729 21689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:02.547765 21676 server_base.cc:1034] running on GCE node
I20260806 22:34:02.548677 21676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:02.550082 21676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:02.551307 21676 hybrid_clock.cc:648] HybridClock initialized: now 1786055642551224 us; error 72 us; skew 500 ppm
I20260806 22:34:02.551685 21676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:02 dist-test-slave-h3ft krb5kdc[21149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055642, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:02.555943 21676 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:34:02.557930 21676 webserver.cc:460] Webserver started at http://127.15.198.131:33481/ using document root <none> and password file <none>
I20260806 22:34:02.558868 21676 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:02.559062 21676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:02.559439 21676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:02.563410 21676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "b26a28d30ad64c68b919da83b28b7e59"
format_stamp: "Formatted at 2026-08-06 22:34:02 on dist-test-slave-h3ft"
server_key: "f1303f12430b807729588627086925d0"
server_key_iv: "18f41a49644304753e34791731878ead"
server_key_version: "encryptionkey@0"
I20260806 22:34:02.564421 21676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "b26a28d30ad64c68b919da83b28b7e59"
format_stamp: "Formatted at 2026-08-06 22:34:02 on dist-test-slave-h3ft"
server_key: "f1303f12430b807729588627086925d0"
server_key_iv: "18f41a49644304753e34791731878ead"
server_key_version: "encryptionkey@0"
I20260806 22:34:02.571348 21676 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 22:34:02.576356 21703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:02.578871 21676 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:34:02.579295 21676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "b26a28d30ad64c68b919da83b28b7e59"
format_stamp: "Formatted at 2026-08-06 22:34:02 on dist-test-slave-h3ft"
server_key: "f1303f12430b807729588627086925d0"
server_key_iv: "18f41a49644304753e34791731878ead"
server_key_version: "encryptionkey@0"
I20260806 22:34:02.579591 21676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:02.609266 21676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:02.616046 21676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:02.616482 21676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:02.618141 21676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:02.620460 21676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:02.620576 21676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:02.620783 21676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:02.620858 21676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:02.666049 21676 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:33263
I20260806 22:34:02.666062 21855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:33263 every 8 connection(s)
I20260806 22:34:02.668088 21676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:34:02.673110 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 21676
I20260806 22:34:02.673374 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:34:02.673947 16154 external_mini_cluster.cc:1428] Setting key db1a15386921aa5d0372ac0d22430ffa
Aug 06 22:34:02 dist-test-slave-h3ft krb5kdc[21149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055642, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:34:02.703277 21864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:02.672240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:41363 (local address 127.15.198.190:43597)
0806 22:34:02.673452 (+ 1212us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:02.673462 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:02.675210 (+ 1748us) server_negotiation.cc:407] Connection header received
0806 22:34:02.678273 (+ 3063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:02.678280 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:02.678449 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:02.678600 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:02.681261 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:02.682190 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:02.684719 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:02.685264 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:02.685455 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:02.692462 (+ 7007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:02.692470 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:02.692475 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:02.692617 (+ 142us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:02.696753 (+ 4136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:02.698579 (+ 1826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:02.698586 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:02.698589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:02.698817 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:02.699968 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:02.699974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:02.699976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:02.700458 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:02.700647 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:02.702443 (+ 1796us) server_negotiation.cc:299] Negotiation successful
0806 22:34:02.702758 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1013,"thread_start_us":786,"threads_started":1}
I20260806 22:34:02.705387 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:02.673634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:43597 (local address 127.15.198.131:41363)
0806 22:34:02.674900 (+ 1266us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:02.674970 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:02.677862 (+ 2892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:02.679109 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:02.679123 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:02.679940 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:02.680953 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:02.680983 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:02.682458 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:02.682463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:02.684425 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:02.684451 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:02.685615 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:02.685624 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:02.685982 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:02.687678 (+ 1696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:02.687750 (+ 72us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:02.692078 (+ 4328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:02.697478 (+ 5400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:02.697492 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:02.697517 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:02.698338 (+ 821us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:02.699139 (+ 801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:02.699145 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:02.699148 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:02.699483 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:02.700814 (+ 1331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:02.700826 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:02.701415 (+ 589us) client_negotiation.cc:769] Sending connection context
0806 22:34:02.701974 (+ 559us) client_negotiation.cc:240] Negotiation successful
0806 22:34:02.702763 (+ 789us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"thread_start_us":503,"threads_started":1}
I20260806 22:34:02.707726 21860 heartbeater.cc:347] Connected to a master server at 127.15.198.190:43597
I20260806 22:34:02.708350 21860 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:02.709509 21860 heartbeater.cc:511] Master 127.15.198.190:43597 requested a full tablet report, sending...
I20260806 22:34:02.712358 21211 ts_manager.cc:194] Registered new tserver with Master: b26a28d30ad64c68b919da83b28b7e59 (127.15.198.131:33263)
I20260806 22:34:02.714303 21211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:41363
I20260806 22:34:02.721961 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:34:02 dist-test-slave-h3ft krb5kdc[21149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:34:02 dist-test-slave-h3ft krb5kdc[21149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:34:02.805526 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:02.783304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59894 (local address 127.15.198.129:33139)
0806 22:34:02.784252 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:02.784262 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:02.784299 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:34:02.784397 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:02.784406 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:02.784724 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:02.784952 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:02.787135 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:02.788422 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:02.790148 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:02.791039 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:02.791185 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:02.797077 (+ 5892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:02.797106 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:02.797130 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:02.797210 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:02.801208 (+ 3998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:02.802373 (+ 1165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:02.802384 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:02.802386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:02.802599 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:02.803261 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:02.803267 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:02.803269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:02.803900 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:02.804236 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:02.804675 (+ 439us) server_negotiation.cc:299] Negotiation successful
0806 22:34:02.804971 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":509,"threads_started":1}
Aug 06 22:34:02 dist-test-slave-h3ft krb5kdc[21149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:34:02.832284 21864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:02.812228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38256 (local address 127.15.198.190:43597)
0806 22:34:02.812761 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:02.812772 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:02.813175 (+ 403us) server_negotiation.cc:407] Connection header received
0806 22:34:02.813678 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:02.813683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:02.813783 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:02.813900 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:02.815851 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:02.816825 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:02.818485 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:02.819138 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:02.819274 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:02.824355 (+ 5081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:02.824363 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:02.824368 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:02.824423 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:02.828314 (+ 3891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:02.829399 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:02.829404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:02.829407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:02.829612 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:02.830112 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:02.830116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:02.830118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:02.830764 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:02.831151 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:02.831577 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 22:34:02.831893 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334}
W20260806 22:34:02.833218 21211 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38256
I20260806 22:34:02.834786 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 21302
I20260806 22:34:02.854344 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:02.836320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59910 (local address 127.15.198.129:33139)
0806 22:34:02.836650 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:02.836657 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:02.836799 (+ 142us) server_negotiation.cc:407] Connection header received
0806 22:34:02.837103 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:02.837116 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:02.837233 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:02.837387 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:02.838927 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:02.839894 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:02.841344 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:02.841851 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:02.842023 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:02.845156 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:02.845161 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:02.845165 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:02.845216 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:02.850099 (+ 4883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:02.851516 (+ 1417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:02.851523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:02.851526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:02.851794 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:02.852395 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:02.852400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:02.852402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:02.852807 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:02.853100 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:02.853573 (+ 473us) server_negotiation.cc:299] Negotiation successful
0806 22:34:02.853843 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260806 22:34:02.855297 21475 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:02.956847 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 21302
I20260806 22:34:02.976078 21665 heartbeater.cc:503] Master 127.15.198.190:43597 was elected leader, sending a full tablet report...
I20260806 22:34:02.995860 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 21493
Aug 06 22:34:03 dist-test-slave-h3ft krb5kdc[21149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.130@KRBTEST.COM
I20260806 22:34:03.018700 21876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:02.997611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55396 (local address 127.15.198.130:46661)
0806 22:34:02.998436 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:02.998445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:02.998482 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:34:02.998566 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:02.998575 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:02.998876 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:02.999145 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:03.000673 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:03.001872 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:03.003584 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:03.004290 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:03.004411 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:03.010242 (+ 5831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:03.010266 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:03.010290 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:03.010358 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:03.014394 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:03.015679 (+ 1285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:03.015691 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:03.015694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:03.015910 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:03.016449 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:03.016454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:03.016456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:03.016968 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:03.017433 (+ 465us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:03.017811 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 22:34:03.018194 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":465,"threads_started":1}
I20260806 22:34:03.020777 21660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:03.121559 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 21493
I20260806 22:34:03.158967 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 21676
Aug 06 22:34:03 dist-test-slave-h3ft krb5kdc[21149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.131@KRBTEST.COM
I20260806 22:34:03.182394 21878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:03.160609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57404 (local address 127.15.198.131:33263)
0806 22:34:03.161306 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:03.161315 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:03.161384 (+ 69us) server_negotiation.cc:407] Connection header received
0806 22:34:03.161473 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:03.161481 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:03.161784 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:03.162041 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:03.163820 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:03.164967 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:03.166651 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:03.167491 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:03.167691 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:03.173847 (+ 6156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:03.173874 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:03.173898 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:03.173968 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:03.178053 (+ 4085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:03.179336 (+ 1283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:03.179347 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:03.179349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:03.179547 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:03.180194 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:03.180199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:03.180202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:03.180722 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:03.181047 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:03.181547 (+ 500us) server_negotiation.cc:299] Negotiation successful
0806 22:34:03.181928 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":332,"threads_started":1}
I20260806 22:34:03.184646 21847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:03.283901 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 21676
I20260806 22:34:03.313647 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 21164
I20260806 22:34:03.315711 21256 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:03.446820 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 21164
2026-08-06T22:34:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (5347 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:34:03 dist-test-slave-h3ft krb5kdc[21885](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:34:03 dist-test-slave-h3ft krb5kdc[21885](info): set up 2 sockets
Aug 06 22:34:03 dist-test-slave-h3ft krb5kdc[21885](info): commencing operation
krb5kdc: starting...
W20260806 22:34:04.855057 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.344s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:34:05 dist-test-slave-h3ft krb5kdc[21885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:34:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:34:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:34:05.234400 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:42933
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:36361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:42933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:05.822906 21904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:05.824014 21904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:05.824239 21904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:05.835726 21904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:34:05.835939 21904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:05.836004 21904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:05.836130 21904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:34:05.836221 21904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:34:05.850478 21904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:42933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:42933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:05.852864 21904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:05.854825 21904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:05.871945 21909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:05.873071 21912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:05.873096 21910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:05.873556 21904 server_base.cc:1034] running on GCE node
I20260806 22:34:05.874769 21904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:05.877249 21904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:05.878615 21904 hybrid_clock.cc:648] HybridClock initialized: now 1786055645878553 us; error 70 us; skew 500 ppm
I20260806 22:34:05.879377 21904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:05 dist-test-slave-h3ft krb5kdc[21885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055645, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:05.884011 21904 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:34:05.885951 21904 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:05.886161 21904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:05.886634 21904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:05.891829 21904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "2439ffa943594e46903a30d0e1c21316"
format_stamp: "Formatted at 2026-08-06 22:34:05 on dist-test-slave-h3ft"
server_key: "db871456bab79b2cdcd4908ef555dbf4"
server_key_iv: "59c8f4f620a95d9f05ec877865ff3e3d"
server_key_version: "encryptionkey@0"
I20260806 22:34:05.893097 21904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "2439ffa943594e46903a30d0e1c21316"
format_stamp: "Formatted at 2026-08-06 22:34:05 on dist-test-slave-h3ft"
server_key: "db871456bab79b2cdcd4908ef555dbf4"
server_key_iv: "59c8f4f620a95d9f05ec877865ff3e3d"
server_key_version: "encryptionkey@0"
I20260806 22:34:05.901336 21904 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.008s sys 0.000s
I20260806 22:34:05.907804 21923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:05.910364 21904 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260806 22:34:05.910794 21904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "2439ffa943594e46903a30d0e1c21316"
format_stamp: "Formatted at 2026-08-06 22:34:05 on dist-test-slave-h3ft"
server_key: "db871456bab79b2cdcd4908ef555dbf4"
server_key_iv: "59c8f4f620a95d9f05ec877865ff3e3d"
server_key_version: "encryptionkey@0"
I20260806 22:34:05.911221 21904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:05.935292 21904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:05.943621 21904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:05.944046 21904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:05.973774 21995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:42933 every 8 connection(s)
I20260806 22:34:05.973789 21904 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:42933
I20260806 22:34:05.975307 21904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:34:05.981213 21996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:05.982326 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 21904
I20260806 22:34:05.982589 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:34:05.983564 16154 external_mini_cluster.cc:1428] Setting key f1ad3e7c909db106f6febaa4df7ff1de
I20260806 22:34:06.002177 21996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316: Bootstrap starting.
Aug 06 22:34:06 dist-test-slave-h3ft krb5kdc[21885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:34:06.007035 21996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:06.008985 21996 log.cc:824] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:06.014585 21996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316: No bootstrap required, opened a new log
I20260806 22:34:06.019630 21999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:05.987440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55216 (local address 127.15.198.190:42933)
0806 22:34:05.988744 (+ 1304us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:05.988767 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:05.988844 (+ 77us) server_negotiation.cc:407] Connection header received
0806 22:34:05.991340 (+ 2496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:05.991397 (+ 57us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:05.992321 (+ 924us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:05.993054 (+ 733us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:05.995103 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:05.997451 (+ 2348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:05.999167 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:06.000630 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:06.000904 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:06.006694 (+ 5790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:06.006712 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:06.006742 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:06.006830 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:06.012086 (+ 5256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:06.013240 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:06.013250 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:06.013262 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:06.013561 (+ 299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:06.014357 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:06.014363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:06.014367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:06.015658 (+ 1291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:06.016091 (+ 433us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:06.016779 (+ 688us) server_negotiation.cc:299] Negotiation successful
0806 22:34:06.017462 (+ 683us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":917,"thread_start_us":593,"threads_started":1}
I20260806 22:34:06.030228 21996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2439ffa943594e46903a30d0e1c21316" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42933 } }
I20260806 22:34:06.031219 21996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:06.031424 21996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2439ffa943594e46903a30d0e1c21316, State: Initialized, Role: FOLLOWER
I20260806 22:34:06.032630 21996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2439ffa943594e46903a30d0e1c21316" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42933 } }
I20260806 22:34:06.033056 21996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:34:06.033245 21996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:34:06.033505 21996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:06.037802 21996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2439ffa943594e46903a30d0e1c21316" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42933 } }
I20260806 22:34:06.038637 21996 leader_election.cc:302] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2439ffa943594e46903a30d0e1c21316; no voters:
I20260806 22:34:06.039676 21996 leader_election.cc:288] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:34:06.039851 22001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:06.041501 22001 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [term 1 LEADER]: Becoming Leader. State: Replica: 2439ffa943594e46903a30d0e1c21316, State: Running, Role: LEADER
I20260806 22:34:06.042200 22001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2439ffa943594e46903a30d0e1c21316" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42933 } }
I20260806 22:34:06.042980 21996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:34:06.046059 22004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2439ffa943594e46903a30d0e1c21316" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2439ffa943594e46903a30d0e1c21316" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42933 } } }
I20260806 22:34:06.045918 22005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2439ffa943594e46903a30d0e1c21316. Latest consensus state: current_term: 1 leader_uuid: "2439ffa943594e46903a30d0e1c21316" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2439ffa943594e46903a30d0e1c21316" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42933 } } }
I20260806 22:34:06.046451 22004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:06.046448 22005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:06.047663 22011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:34:06.055310 22011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:34:06.072017 22011 catalog_manager.cc:1347] Generated new cluster ID: 3dbed53c1044471b918df2f2dc1dd431
I20260806 22:34:06.072202 22011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:34:06.097110 22011 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:34:06.099892 22011 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:34:06.116763 22011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316: Generated new TSK 0
I20260806 22:34:06.118615 22011 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
I20260806 22:34:06.201645 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42933
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:36361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:06.666014 22032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:06.666496 22032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:06.666975 22032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:06.677412 22032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:06.677728 22032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:06.677946 22032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:34:06.690793 22032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:06.692906 22032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:06.694847 22032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:06.713069 22040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:06.713045 22041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:06.712874 22044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:06.714751 22032 server_base.cc:1034] running on GCE node
I20260806 22:34:06.715821 22032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:06.717370 22032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:06.718695 22032 hybrid_clock.cc:648] HybridClock initialized: now 1786055646718629 us; error 56 us; skew 500 ppm
I20260806 22:34:06.719251 22032 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:06 dist-test-slave-h3ft krb5kdc[21885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055646, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:06.723768 22032 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:34:06.725582 22032 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:06.725721 22032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:06.726106 22032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:06.730633 22032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "8f6cfb3a5f95497fbb04a8b0dcd59da7"
format_stamp: "Formatted at 2026-08-06 22:34:06 on dist-test-slave-h3ft"
server_key: "f85ee593d1fda4e41dcde5668b72c9f5"
server_key_iv: "717f28cfd4ccff401710b84a24beca08"
server_key_version: "encryptionkey@0"
I20260806 22:34:06.731999 22032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "8f6cfb3a5f95497fbb04a8b0dcd59da7"
format_stamp: "Formatted at 2026-08-06 22:34:06 on dist-test-slave-h3ft"
server_key: "f85ee593d1fda4e41dcde5668b72c9f5"
server_key_iv: "717f28cfd4ccff401710b84a24beca08"
server_key_version: "encryptionkey@0"
I20260806 22:34:06.739945 22032 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 22:34:06.745317 22052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:06.747457 22032 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:34:06.747756 22032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "8f6cfb3a5f95497fbb04a8b0dcd59da7"
format_stamp: "Formatted at 2026-08-06 22:34:06 on dist-test-slave-h3ft"
server_key: "f85ee593d1fda4e41dcde5668b72c9f5"
server_key_iv: "717f28cfd4ccff401710b84a24beca08"
server_key_version: "encryptionkey@0"
I20260806 22:34:06.748106 22032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:06.774542 22032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:06.781211 22032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:06.781628 22032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:06.783078 22032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:06.785079 22032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:06.785185 22032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:06.785351 22032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:06.785447 22032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:06.829653 22032 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:45577
I20260806 22:34:06.829772 22212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:45577 every 8 connection(s)
I20260806 22:34:06.831470 22032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:34:06.842135 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 22032
I20260806 22:34:06.842440 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:34:06.843626 16154 external_mini_cluster.cc:1428] Setting key d274cfb9fbd78ece37e7cf4ca158e3df
Aug 06 22:34:06 dist-test-slave-h3ft krb5kdc[21885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055646, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:34:06.865525 22220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:06.834977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:48145 (local address 127.15.198.190:42933)
0806 22:34:06.835672 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:06.835678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:06.837245 (+ 1567us) server_negotiation.cc:407] Connection header received
0806 22:34:06.839759 (+ 2514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:06.839766 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:06.839891 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:06.840051 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:06.842203 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:06.843365 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:06.846721 (+ 3356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:06.847505 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:06.847687 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:06.854790 (+ 7103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:06.854797 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:06.854802 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:06.854858 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:06.859775 (+ 4917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:06.861544 (+ 1769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:06.861554 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:06.861559 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:06.861817 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:06.862614 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:06.862619 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:06.862622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:06.863143 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:06.863287 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:06.864788 (+ 1501us) server_negotiation.cc:299] Negotiation successful
0806 22:34:06.865155 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":287,"threads_started":1}
I20260806 22:34:06.867532 22221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:06.835826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:42933 (local address 127.15.198.129:48145)
0806 22:34:06.836990 (+ 1164us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:06.837061 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:06.839321 (+ 2260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:06.840377 (+ 1056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:06.840389 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:06.841117 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:06.841935 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:06.841958 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:06.843574 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:06.843580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:06.846488 (+ 2908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:06.846515 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:06.847832 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:06.847841 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:06.848210 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:06.849213 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:06.849282 (+ 69us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:06.854130 (+ 4848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:06.860122 (+ 5992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:06.860142 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:06.860177 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:06.861282 (+ 1105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:06.862097 (+ 815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:06.862102 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:06.862105 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:06.862451 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:06.863382 (+ 931us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:06.863398 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:06.864139 (+ 741us) client_negotiation.cc:769] Sending connection context
0806 22:34:06.864601 (+ 462us) client_negotiation.cc:240] Negotiation successful
0806 22:34:06.865436 (+ 835us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"thread_start_us":472,"threads_started":1}
I20260806 22:34:06.870507 22213 heartbeater.cc:347] Connected to a master server at 127.15.198.190:42933
I20260806 22:34:06.871047 22213 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:06.872381 22213 heartbeater.cc:511] Master 127.15.198.190:42933 requested a full tablet report, sending...
I20260806 22:34:06.876091 21952 ts_manager.cc:194] Registered new tserver with Master: 8f6cfb3a5f95497fbb04a8b0dcd59da7 (127.15.198.129:45577)
I20260806 22:34:06.878234 21952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:48145
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
I20260806 22:34:06.924664 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42933
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:36361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:07.387714 22230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:07.388168 22230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:07.388363 22230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:07.396986 22230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:07.397231 22230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:07.397462 22230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:34:07.409981 22230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:07.412189 22230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:07.414145 22230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:07.432125 22240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:07.432144 22242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:07.432185 22239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:07.433300 22230 server_base.cc:1034] running on GCE node
I20260806 22:34:07.434293 22230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:07.435752 22230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:07.437050 22230 hybrid_clock.cc:648] HybridClock initialized: now 1786055647436997 us; error 48 us; skew 500 ppm
I20260806 22:34:07.437533 22230 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:07 dist-test-slave-h3ft krb5kdc[21885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055647, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:07.442109 22230 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:34:07.444224 22230 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:07.444427 22230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:07.444983 22230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:07.450198 22230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "f3eb8e975ddd43cfb1ab3212b95a35b7"
format_stamp: "Formatted at 2026-08-06 22:34:07 on dist-test-slave-h3ft"
server_key: "201585ad1ce66e92642758533f0fb651"
server_key_iv: "f0c211db3113c2971674eb9a66c4a0d4"
server_key_version: "encryptionkey@0"
I20260806 22:34:07.451630 22230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "f3eb8e975ddd43cfb1ab3212b95a35b7"
format_stamp: "Formatted at 2026-08-06 22:34:07 on dist-test-slave-h3ft"
server_key: "201585ad1ce66e92642758533f0fb651"
server_key_iv: "f0c211db3113c2971674eb9a66c4a0d4"
server_key_version: "encryptionkey@0"
I20260806 22:34:07.460696 22230 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260806 22:34:07.466437 22248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:07.468806 22230 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260806 22:34:07.469110 22230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "f3eb8e975ddd43cfb1ab3212b95a35b7"
format_stamp: "Formatted at 2026-08-06 22:34:07 on dist-test-slave-h3ft"
server_key: "201585ad1ce66e92642758533f0fb651"
server_key_iv: "f0c211db3113c2971674eb9a66c4a0d4"
server_key_version: "encryptionkey@0"
I20260806 22:34:07.469444 22230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:07.488389 22230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:07.494534 22230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:07.494935 22230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:07.496405 22230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:07.498451 22230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:07.498581 22230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:07.498781 22230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:07.498878 22230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:07.547083 22230 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:38983
I20260806 22:34:07.547188 22403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:38983 every 8 connection(s)
I20260806 22:34:07.548290 22230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:34:07.553144 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 22230
I20260806 22:34:07.553390 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:34:07.554127 16154 external_mini_cluster.cc:1428] Setting key 0a3faf8736cc44b84e0d727915259c7b
Aug 06 22:34:07 dist-test-slave-h3ft krb5kdc[21885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055647, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:34:07.578295 22409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:07.551681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:40835 (local address 127.15.198.190:42933)
0806 22:34:07.552392 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:07.552398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:07.553895 (+ 1497us) server_negotiation.cc:407] Connection header received
0806 22:34:07.556758 (+ 2863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:07.556764 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:07.556872 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:07.556994 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:07.559150 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:07.560631 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:07.562899 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:07.563912 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:07.564099 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:07.568957 (+ 4858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:07.568965 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:07.568969 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:07.569022 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:07.572991 (+ 3969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:07.574523 (+ 1532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:07.574531 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:07.574534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:07.574731 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:07.575550 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:07.575555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:07.575557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:07.576080 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:07.576250 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:07.577648 (+ 1398us) server_negotiation.cc:299] Negotiation successful
0806 22:34:07.577944 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":401,"threads_started":1}
I20260806 22:34:07.580348 22410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:07.552294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:42933 (local address 127.15.198.130:40835)
0806 22:34:07.553655 (+ 1361us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:07.553715 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:07.556292 (+ 2577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:07.557356 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:07.557368 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:07.558090 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:07.558866 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:07.558886 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:07.560857 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:07.560862 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:07.562696 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:07.562714 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:07.564239 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:07.564247 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:07.564566 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:07.565530 (+ 964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:07.565571 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:07.568522 (+ 2951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:07.573301 (+ 4779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:07.573321 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:07.573364 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:07.574305 (+ 941us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:07.574895 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:07.574903 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:07.574910 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:07.575288 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:07.576288 (+ 1000us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:07.576301 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:07.576996 (+ 695us) client_negotiation.cc:769] Sending connection context
0806 22:34:07.577437 (+ 441us) client_negotiation.cc:240] Negotiation successful
0806 22:34:07.578263 (+ 826us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":482,"threads_started":1}
I20260806 22:34:07.582752 22404 heartbeater.cc:347] Connected to a master server at 127.15.198.190:42933
I20260806 22:34:07.583377 22404 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:07.584494 22404 heartbeater.cc:511] Master 127.15.198.190:42933 requested a full tablet report, sending...
I20260806 22:34:07.586918 21952 ts_manager.cc:194] Registered new tserver with Master: f3eb8e975ddd43cfb1ab3212b95a35b7 (127.15.198.130:38983)
I20260806 22:34:07.588145 21952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:40835
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
I20260806 22:34:07.632494 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42933
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:36361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:34:07.883091 22213 heartbeater.cc:503] Master 127.15.198.190:42933 was elected leader, sending a full tablet report...
W20260806 22:34:08.110227 22415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:08.110646 22415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:08.110769 22415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:08.120306 22415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:08.120500 22415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:08.120645 22415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:34:08.133160 22415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:08.135389 22415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:08.137317 22415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:08.155634 22426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:08.155663 22421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:08.155582 22420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:08.156594 22415 server_base.cc:1034] running on GCE node
I20260806 22:34:08.157407 22415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:08.159409 22415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:08.160712 22415 hybrid_clock.cc:648] HybridClock initialized: now 1786055648160633 us; error 74 us; skew 500 ppm
I20260806 22:34:08.161204 22415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:08 dist-test-slave-h3ft krb5kdc[21885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055648, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:08.165553 22415 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:34:08.167341 22415 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:08.167514 22415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:08.167927 22415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:08.171849 22415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "86bfe052501a49da964600c726267fcb"
format_stamp: "Formatted at 2026-08-06 22:34:08 on dist-test-slave-h3ft"
server_key: "6a6b2860bf655dea6233fa6246d715ba"
server_key_iv: "2f8a21626fa9268680c577f0a4a47522"
server_key_version: "encryptionkey@0"
I20260806 22:34:08.172956 22415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "86bfe052501a49da964600c726267fcb"
format_stamp: "Formatted at 2026-08-06 22:34:08 on dist-test-slave-h3ft"
server_key: "6a6b2860bf655dea6233fa6246d715ba"
server_key_iv: "2f8a21626fa9268680c577f0a4a47522"
server_key_version: "encryptionkey@0"
I20260806 22:34:08.180222 22415 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260806 22:34:08.185334 22436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:08.187575 22415 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:34:08.187848 22415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "86bfe052501a49da964600c726267fcb"
format_stamp: "Formatted at 2026-08-06 22:34:08 on dist-test-slave-h3ft"
server_key: "6a6b2860bf655dea6233fa6246d715ba"
server_key_iv: "2f8a21626fa9268680c577f0a4a47522"
server_key_version: "encryptionkey@0"
I20260806 22:34:08.188221 22415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:08.228315 22415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:08.234500 22415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:08.234983 22415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:08.236732 22415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:08.238744 22415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:08.238898 22415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:08.239084 22415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:08.239197 22415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:08.282351 22415 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:36643
I20260806 22:34:08.282414 22606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:36643 every 8 connection(s)
I20260806 22:34:08.283615 22415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:34:08.292141 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 22415
I20260806 22:34:08.292404 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:34:08.293110 16154 external_mini_cluster.cc:1428] Setting key 4041024a954f77c04819d0486cfd3f90
Aug 06 22:34:08 dist-test-slave-h3ft krb5kdc[21885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055648, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:34:08.312542 22613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:08.287187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:39619 (local address 127.15.198.190:42933)
0806 22:34:08.287793 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:08.287802 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:08.289097 (+ 1295us) server_negotiation.cc:407] Connection header received
0806 22:34:08.291246 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:08.291252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:08.291364 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:08.291491 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:08.294370 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.295496 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.298085 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.298601 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.298723 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:08.303500 (+ 4777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:08.303506 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:08.303509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:08.303555 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:08.307430 (+ 3875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:08.308747 (+ 1317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:08.308753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:08.308755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:08.309020 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:08.309863 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:08.309870 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:08.309875 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:08.310392 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:08.310595 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:08.311911 (+ 1316us) server_negotiation.cc:299] Negotiation successful
0806 22:34:08.312169 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":277,"threads_started":1}
I20260806 22:34:08.313915 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:08.287926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:42933 (local address 127.15.198.131:39619)
0806 22:34:08.288863 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:08.288917 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:08.290838 (+ 1921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:08.291893 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:08.291909 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:08.292994 (+ 1085us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:08.294111 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:08.294137 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.295757 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.295765 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:08.297861 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:08.297878 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.298858 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.298864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:08.299251 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:08.300119 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:08.300148 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:08.303189 (+ 3041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:08.307637 (+ 4448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:08.307648 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:08.307691 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:08.308571 (+ 880us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:08.309201 (+ 630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:08.309208 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:08.309215 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:08.309603 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:08.310588 (+ 985us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:08.310596 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:08.311395 (+ 799us) client_negotiation.cc:769] Sending connection context
0806 22:34:08.311775 (+ 380us) client_negotiation.cc:240] Negotiation successful
0806 22:34:08.312358 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":397,"threads_started":1}
I20260806 22:34:08.316145 22610 heartbeater.cc:347] Connected to a master server at 127.15.198.190:42933
I20260806 22:34:08.316634 22610 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:08.317631 22610 heartbeater.cc:511] Master 127.15.198.190:42933 requested a full tablet report, sending...
I20260806 22:34:08.319705 21952 ts_manager.cc:194] Registered new tserver with Master: 86bfe052501a49da964600c726267fcb (127.15.198.131:36643)
I20260806 22:34:08.320814 21952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:39619
I20260806 22:34:08.324316 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:34:08.346470 22613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:08.330658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55218 (local address 127.15.198.190:42933)
0806 22:34:08.330911 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:08.330917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:08.331573 (+ 656us) server_negotiation.cc:407] Connection header received
0806 22:34:08.331868 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:08.331873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:08.331971 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:08.332095 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:08.333917 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.334765 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.336278 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.336901 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.337034 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:08.340010 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:08.340016 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:08.340020 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:08.340071 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:08.343288 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:08.344224 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:08.344228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:08.344231 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:08.344383 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:08.344853 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:08.344857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:08.344859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:08.345284 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:08.345527 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:08.345903 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 22:34:08.346142 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 22:34:08.352591 21952 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:34:08.362089 21952 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:34:08.398957 22633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:08.384041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:38983 (local address 127.0.0.1:56500)
0806 22:34:08.387747 (+ 3706us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:08.387789 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:08.387984 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:08.389836 (+ 1852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:08.389842 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:08.389878 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:08.390363 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:08.390380 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.392588 (+ 2208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.392594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:08.394981 (+ 2387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:08.394995 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.397482 (+ 2487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.397494 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:08.397793 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:08.397895 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 22:34:08.398046 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 22:34:08.398339 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3517,"mutex_wait_us":30,"thread_start_us":555,"threads_started":1}
I20260806 22:34:08.399874 22634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:08.381749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50592 (local address 127.15.198.131:36643)
0806 22:34:08.385731 (+ 3982us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:08.385749 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:08.385805 (+ 56us) server_negotiation.cc:407] Connection header received
0806 22:34:08.386006 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:08.386021 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:08.386455 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:08.386783 (+ 328us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:08.389521 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.391560 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.395911 (+ 4351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.397625 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.397851 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:08.398126 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:08.399155 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0806 22:34:08.399415 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3680,"thread_start_us":3428,"threads_started":1}
I20260806 22:34:08.400413 22635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:08.381040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:36643 (local address 127.0.0.1:50592)
0806 22:34:08.385006 (+ 3966us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:08.385043 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:08.385213 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:08.386818 (+ 1605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:08.386827 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:08.386865 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:08.389181 (+ 2316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:08.389203 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.392167 (+ 2964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.392180 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:08.395681 (+ 3501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:08.395696 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.398299 (+ 2603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.398306 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:08.398499 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:08.398555 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 22:34:08.398645 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 22:34:08.398804 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3772,"mutex_wait_us":789,"thread_start_us":6341,"threads_started":1}
I20260806 22:34:08.400971 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:08.384891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56500 (local address 127.15.198.130:38983)
0806 22:34:08.388735 (+ 3844us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:08.388744 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:08.388776 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:34:08.388890 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:08.388898 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:08.389162 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:08.389453 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:08.390615 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.392331 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.395252 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.397168 (+ 1916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.399921 (+ 2753us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:08.400304 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:08.400390 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 22:34:08.400644 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3632,"thread_start_us":3433,"threads_started":1}
I20260806 22:34:08.402781 22628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:08.380795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:45577 (local address 127.0.0.1:49920)
0806 22:34:08.381994 (+ 1199us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:08.382074 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:08.382347 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:08.394995 (+ 12648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:08.395029 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:08.395066 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:08.395592 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:08.395606 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.397234 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.397241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:08.399609 (+ 2368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:08.399625 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.401883 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.401890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:08.402100 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:08.402149 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 22:34:08.402255 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 22:34:08.402420 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":576,"threads_started":1}
I20260806 22:34:08.403038 22632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:08.380984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49920 (local address 127.15.198.129:45577)
0806 22:34:08.394063 (+ 13079us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:08.394076 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:08.394119 (+ 43us) server_negotiation.cc:407] Connection header received
0806 22:34:08.394259 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:08.394269 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:08.394694 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:08.394981 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:08.395849 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.397045 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.399847 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.401611 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.401864 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:08.402067 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:08.402391 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 22:34:08.402625 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12844,"thread_start_us":12474,"threads_started":1}
I20260806 22:34:08.406113 22314 tablet_service.cc:1467] Processing CreateTablet for tablet 54457c135ad14850a8acbc4cd34de53d (DEFAULT_TABLE table=test-table [id=1aa3b7c6109047feba87d4dbce0e316f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:08.406113 22511 tablet_service.cc:1467] Processing CreateTablet for tablet 54457c135ad14850a8acbc4cd34de53d (DEFAULT_TABLE table=test-table [id=1aa3b7c6109047feba87d4dbce0e316f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:08.407506 22117 tablet_service.cc:1467] Processing CreateTablet for tablet 54457c135ad14850a8acbc4cd34de53d (DEFAULT_TABLE table=test-table [id=1aa3b7c6109047feba87d4dbce0e316f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:08.408249 22314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54457c135ad14850a8acbc4cd34de53d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:08.408241 22511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54457c135ad14850a8acbc4cd34de53d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:08.409049 22117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54457c135ad14850a8acbc4cd34de53d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:08.425374 22637 tablet_bootstrap.cc:492] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7: Bootstrap starting.
I20260806 22:34:08.429947 22637 tablet_bootstrap.cc:654] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:08.431253 22639 tablet_bootstrap.cc:492] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7: Bootstrap starting.
I20260806 22:34:08.431851 22637 log.cc:824] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:08.432667 22638 tablet_bootstrap.cc:492] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb: Bootstrap starting.
I20260806 22:34:08.436971 22637 tablet_bootstrap.cc:492] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7: No bootstrap required, opened a new log
I20260806 22:34:08.437434 22637 ts_tablet_manager.cc:1397] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260806 22:34:08.437494 22639 tablet_bootstrap.cc:654] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:08.438294 22638 tablet_bootstrap.cc:654] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:08.438915 22639 log.cc:824] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:08.439801 22638 log.cc:824] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:08.443037 22639 tablet_bootstrap.cc:492] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7: No bootstrap required, opened a new log
I20260806 22:34:08.443455 22639 ts_tablet_manager.cc:1397] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260806 22:34:08.443750 22638 tablet_bootstrap.cc:492] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb: No bootstrap required, opened a new log
I20260806 22:34:08.444185 22638 ts_tablet_manager.cc:1397] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260806 22:34:08.450696 22637 raft_consensus.cc:348] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6cfb3a5f95497fbb04a8b0dcd59da7" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45577 } } peers { permanent_uuid: "86bfe052501a49da964600c726267fcb" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36643 } } peers { permanent_uuid: "f3eb8e975ddd43cfb1ab3212b95a35b7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38983 } }
I20260806 22:34:08.451471 22637 raft_consensus.cc:374] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:08.451640 22637 raft_consensus.cc:729] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f6cfb3a5f95497fbb04a8b0dcd59da7, State: Initialized, Role: FOLLOWER
I20260806 22:34:08.452709 22637 consensus_queue.cc:260] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6cfb3a5f95497fbb04a8b0dcd59da7" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45577 } } peers { permanent_uuid: "86bfe052501a49da964600c726267fcb" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36643 } } peers { permanent_uuid: "f3eb8e975ddd43cfb1ab3212b95a35b7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38983 } }
I20260806 22:34:08.454633 22639 raft_consensus.cc:348] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6cfb3a5f95497fbb04a8b0dcd59da7" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45577 } } peers { permanent_uuid: "86bfe052501a49da964600c726267fcb" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36643 } } peers { permanent_uuid: "f3eb8e975ddd43cfb1ab3212b95a35b7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38983 } }
I20260806 22:34:08.455324 22639 raft_consensus.cc:374] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:08.455444 22639 raft_consensus.cc:729] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3eb8e975ddd43cfb1ab3212b95a35b7, State: Initialized, Role: FOLLOWER
I20260806 22:34:08.455709 22637 ts_tablet_manager.cc:1428] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7: Time spent starting tablet: real 0.018s user 0.020s sys 0.000s
I20260806 22:34:08.456473 22639 consensus_queue.cc:260] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6cfb3a5f95497fbb04a8b0dcd59da7" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45577 } } peers { permanent_uuid: "86bfe052501a49da964600c726267fcb" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36643 } } peers { permanent_uuid: "f3eb8e975ddd43cfb1ab3212b95a35b7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38983 } }
I20260806 22:34:08.457085 22638 raft_consensus.cc:348] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6cfb3a5f95497fbb04a8b0dcd59da7" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45577 } } peers { permanent_uuid: "86bfe052501a49da964600c726267fcb" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36643 } } peers { permanent_uuid: "f3eb8e975ddd43cfb1ab3212b95a35b7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38983 } }
I20260806 22:34:08.457695 22638 raft_consensus.cc:374] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:08.457825 22638 raft_consensus.cc:729] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86bfe052501a49da964600c726267fcb, State: Initialized, Role: FOLLOWER
I20260806 22:34:08.458707 22638 consensus_queue.cc:260] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6cfb3a5f95497fbb04a8b0dcd59da7" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45577 } } peers { permanent_uuid: "86bfe052501a49da964600c726267fcb" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36643 } } peers { permanent_uuid: "f3eb8e975ddd43cfb1ab3212b95a35b7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38983 } }
I20260806 22:34:08.460325 22610 heartbeater.cc:503] Master 127.15.198.190:42933 was elected leader, sending a full tablet report...
I20260806 22:34:08.461249 22638 ts_tablet_manager.cc:1428] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260806 22:34:08.462815 22404 heartbeater.cc:503] Master 127.15.198.190:42933 was elected leader, sending a full tablet report...
I20260806 22:34:08.464279 22639 ts_tablet_manager.cc:1428] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7: Time spent starting tablet: real 0.021s user 0.015s sys 0.005s
I20260806 22:34:08.470121 22648 raft_consensus.cc:482] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:34:08.470530 22648 raft_consensus.cc:504] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6cfb3a5f95497fbb04a8b0dcd59da7" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45577 } } peers { permanent_uuid: "86bfe052501a49da964600c726267fcb" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36643 } } peers { permanent_uuid: "f3eb8e975ddd43cfb1ab3212b95a35b7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38983 } }
I20260806 22:34:08.472951 22648 leader_election.cc:288] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f6cfb3a5f95497fbb04a8b0dcd59da7 (127.15.198.129:45577), f3eb8e975ddd43cfb1ab3212b95a35b7 (127.15.198.130:38983)
I20260806 22:34:08.481501 22632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:08.473615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:48169 (local address 127.15.198.129:45577)
0806 22:34:08.474654 (+ 1039us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:08.474662 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:08.474690 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:34:08.474794 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:08.474800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:08.474901 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:08.475096 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:08.475664 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.476588 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.478777 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.480372 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.480554 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:08.480645 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:08.481090 (+ 445us) server_negotiation.cc:299] Negotiation successful
0806 22:34:08.481229 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":824}
I20260806 22:34:08.482182 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:08.473451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:45577 (local address 127.15.198.131:48169)
0806 22:34:08.473757 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:08.473781 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:08.473929 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:08.475137 (+ 1208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:08.475149 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:08.475194 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:08.475480 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:08.475493 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.476753 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.476759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:08.478551 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:08.478567 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.480576 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.480583 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:08.480823 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:08.480865 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 22:34:08.481000 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 22:34:08.481190 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":48,"spinlock_wait_cycles":3840}
I20260806 22:34:08.483062 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:08.474345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:40029 (local address 127.15.198.130:38983)
0806 22:34:08.474568 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:08.474573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:08.475083 (+ 510us) server_negotiation.cc:407] Connection header received
0806 22:34:08.475424 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:08.475430 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:08.475555 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:08.475660 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:08.476250 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.477349 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.480197 (+ 2848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.481551 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.481758 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:08.481828 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:08.482659 (+ 831us) server_negotiation.cc:299] Negotiation successful
0806 22:34:08.482782 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 22:34:08.483287 22649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:08.474027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:38983 (local address 127.15.198.131:40029)
0806 22:34:08.474916 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:08.474944 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:08.475226 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:08.475678 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:08.475683 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:08.475707 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:08.476060 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:08.476069 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.477568 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.477573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:08.479944 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:08.479966 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.482202 (+ 2236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.482208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:08.482453 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:08.482497 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:34:08.482618 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 22:34:08.482783 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"mutex_wait_us":1,"spinlock_wait_cycles":14080,"thread_start_us":1205,"threads_started":1}
I20260806 22:34:08.483872 22146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54457c135ad14850a8acbc4cd34de53d" candidate_uuid: "86bfe052501a49da964600c726267fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f6cfb3a5f95497fbb04a8b0dcd59da7" is_pre_election: true
I20260806 22:34:08.484457 22342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54457c135ad14850a8acbc4cd34de53d" candidate_uuid: "86bfe052501a49da964600c726267fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3eb8e975ddd43cfb1ab3212b95a35b7" is_pre_election: true
I20260806 22:34:08.484774 22146 raft_consensus.cc:2454] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86bfe052501a49da964600c726267fcb in term 0.
I20260806 22:34:08.485116 22342 raft_consensus.cc:2454] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86bfe052501a49da964600c726267fcb in term 0.
I20260806 22:34:08.485937 22441 leader_election.cc:302] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86bfe052501a49da964600c726267fcb, 8f6cfb3a5f95497fbb04a8b0dcd59da7; no voters:
I20260806 22:34:08.487108 22648 raft_consensus.cc:2781] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:34:08.487357 22648 raft_consensus.cc:482] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:34:08.487597 22648 raft_consensus.cc:3037] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:08.490365 22648 raft_consensus.cc:504] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6cfb3a5f95497fbb04a8b0dcd59da7" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45577 } } peers { permanent_uuid: "86bfe052501a49da964600c726267fcb" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36643 } } peers { permanent_uuid: "f3eb8e975ddd43cfb1ab3212b95a35b7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38983 } }
I20260806 22:34:08.491284 22648 leader_election.cc:288] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [CANDIDATE]: Term 1 election: Requested vote from peers 8f6cfb3a5f95497fbb04a8b0dcd59da7 (127.15.198.129:45577), f3eb8e975ddd43cfb1ab3212b95a35b7 (127.15.198.130:38983)
I20260806 22:34:08.492090 22146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54457c135ad14850a8acbc4cd34de53d" candidate_uuid: "86bfe052501a49da964600c726267fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f6cfb3a5f95497fbb04a8b0dcd59da7"
I20260806 22:34:08.492277 22342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54457c135ad14850a8acbc4cd34de53d" candidate_uuid: "86bfe052501a49da964600c726267fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3eb8e975ddd43cfb1ab3212b95a35b7"
I20260806 22:34:08.492672 22146 raft_consensus.cc:3037] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:08.492869 22342 raft_consensus.cc:3037] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:08.495955 22342 raft_consensus.cc:2454] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86bfe052501a49da964600c726267fcb in term 1.
I20260806 22:34:08.495966 22146 raft_consensus.cc:2454] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86bfe052501a49da964600c726267fcb in term 1.
I20260806 22:34:08.496656 22441 leader_election.cc:302] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86bfe052501a49da964600c726267fcb, f3eb8e975ddd43cfb1ab3212b95a35b7; no voters:
I20260806 22:34:08.497155 22648 raft_consensus.cc:2781] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:08.497809 22648 raft_consensus.cc:686] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [term 1 LEADER]: Becoming Leader. State: Replica: 86bfe052501a49da964600c726267fcb, State: Running, Role: LEADER
I20260806 22:34:08.498500 22648 consensus_queue.cc:237] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6cfb3a5f95497fbb04a8b0dcd59da7" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45577 } } peers { permanent_uuid: "86bfe052501a49da964600c726267fcb" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36643 } } peers { permanent_uuid: "f3eb8e975ddd43cfb1ab3212b95a35b7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38983 } }
I20260806 22:34:08.505759 21951 catalog_manager.cc:5495] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb reported cstate change: term changed from 0 to 1, leader changed from <none> to 86bfe052501a49da964600c726267fcb (127.15.198.131). New cstate: current_term: 1 leader_uuid: "86bfe052501a49da964600c726267fcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6cfb3a5f95497fbb04a8b0dcd59da7" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86bfe052501a49da964600c726267fcb" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3eb8e975ddd43cfb1ab3212b95a35b7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38983 } health_report { overall_health: UNKNOWN } } }
I20260806 22:34:08.529443 22634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:08.522478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34000 (local address 127.15.198.131:36643)
0806 22:34:08.522764 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:08.522770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:08.522794 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:34:08.523115 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:08.523120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:08.523229 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:08.523402 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:34:08.523878 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.524755 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.526305 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.526835 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.526969 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:08.527770 (+ 801us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:34:08.528579 (+ 809us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:34:08.528712 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:08.528950 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 22:34:08.529099 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
W20260806 22:34:08.535799 22611 tablet.cc:2369] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:34:08.542668 22146 raft_consensus.cc:1261] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7 [term 1 FOLLOWER]: Refusing update from remote peer 86bfe052501a49da964600c726267fcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:08.542672 22342 raft_consensus.cc:1261] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7 [term 1 FOLLOWER]: Refusing update from remote peer 86bfe052501a49da964600c726267fcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:08.544387 22651 consensus_queue.cc:1035] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3eb8e975ddd43cfb1ab3212b95a35b7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:08.544997 22648 consensus_queue.cc:1035] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f6cfb3a5f95497fbb04a8b0dcd59da7" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:34:08.551038 22405 tablet.cc:2369] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:34:08.590504 21950 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:34:08.591080 21950 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:34:08.597050 21950 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2439ffa943594e46903a30d0e1c21316: Sending DeleteTablet for 3 replicas of tablet 54457c135ad14850a8acbc4cd34de53d
I20260806 22:34:08.599522 22117 tablet_service.cc:1514] Processing DeleteTablet for tablet 54457c135ad14850a8acbc4cd34de53d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:08 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:49920
I20260806 22:34:08.599529 22511 tablet_service.cc:1514] Processing DeleteTablet for tablet 54457c135ad14850a8acbc4cd34de53d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:08 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:50592
I20260806 22:34:08.599488 22314 tablet_service.cc:1514] Processing DeleteTablet for tablet 54457c135ad14850a8acbc4cd34de53d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:08 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:56500
I20260806 22:34:08.603731 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 22032
I20260806 22:34:08.611698 22671 tablet_replica.cc:330] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7: stopping tablet replica
I20260806 22:34:08.612516 22671 raft_consensus.cc:2229] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:34:08.613946 22671 raft_consensus.cc:2258] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:08.617713 22671 ts_tablet_manager.cc:1905] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:08.628090 22668 tablet_replica.cc:330] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb: stopping tablet replica
I20260806 22:34:08.628806 22668 raft_consensus.cc:2229] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:34:08.629716 22668 raft_consensus.cc:2258] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:08.632705 22668 ts_tablet_manager.cc:1905] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:08.633450 22670 tablet_replica.cc:330] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7: stopping tablet replica
I20260806 22:34:08.634071 22670 raft_consensus.cc:2229] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:34:08.634780 22670 raft_consensus.cc:2258] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:08.637692 22670 ts_tablet_manager.cc:1905] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 22:34:08 dist-test-slave-h3ft krb5kdc[21885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:34:08.642524 22668 ts_tablet_manager.cc:1918] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:34:08.642786 22668 log.cc:1196] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/54457c135ad14850a8acbc4cd34de53d
I20260806 22:34:08.643528 22668 ts_tablet_manager.cc:1939] T 54457c135ad14850a8acbc4cd34de53d P 86bfe052501a49da964600c726267fcb: Deleting consensus metadata
I20260806 22:34:08.646464 21924 catalog_manager.cc:4841] TS 86bfe052501a49da964600c726267fcb (127.15.198.131:36643): tablet 54457c135ad14850a8acbc4cd34de53d (table test-table [id=1aa3b7c6109047feba87d4dbce0e316f]) successfully deleted
I20260806 22:34:08.647051 22671 ts_tablet_manager.cc:1918] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:34:08.647426 22671 log.cc:1196] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/54457c135ad14850a8acbc4cd34de53d
I20260806 22:34:08.648173 22671 ts_tablet_manager.cc:1939] T 54457c135ad14850a8acbc4cd34de53d P f3eb8e975ddd43cfb1ab3212b95a35b7: Deleting consensus metadata
I20260806 22:34:08.648212 22670 ts_tablet_manager.cc:1918] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:34:08.648480 22670 log.cc:1196] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/54457c135ad14850a8acbc4cd34de53d
I20260806 22:34:08.649147 22670 ts_tablet_manager.cc:1939] T 54457c135ad14850a8acbc4cd34de53d P 8f6cfb3a5f95497fbb04a8b0dcd59da7: Deleting consensus metadata
I20260806 22:34:08.650894 21936 catalog_manager.cc:4841] TS f3eb8e975ddd43cfb1ab3212b95a35b7 (127.15.198.130:38983): tablet 54457c135ad14850a8acbc4cd34de53d (table test-table [id=1aa3b7c6109047feba87d4dbce0e316f]) successfully deleted
I20260806 22:34:08.651369 21936 catalog_manager.cc:4841] TS 8f6cfb3a5f95497fbb04a8b0dcd59da7 (127.15.198.129:45577): tablet 54457c135ad14850a8acbc4cd34de53d (table test-table [id=1aa3b7c6109047feba87d4dbce0e316f]) successfully deleted
I20260806 22:34:08.655069 22632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:08.605149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34254 (local address 127.15.198.129:45577)
0806 22:34:08.605407 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:08.605414 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:08.628189 (+ 22775us) server_negotiation.cc:407] Connection header received
0806 22:34:08.628792 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:08.628803 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:08.628952 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:08.629096 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:08.632134 (+ 3038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.633712 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.636097 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.636830 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.636972 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:08.644807 (+ 7835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:08.644830 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:08.644856 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:08.644939 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:08.650639 (+ 5700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:08.652222 (+ 1583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:08.652233 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:08.652236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:08.652425 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:08.652999 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:08.653003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:08.653006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:08.653576 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:08.654048 (+ 472us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:08.654285 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 22:34:08.654585 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 22:34:08.655992 22207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:08.798580 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 22032
I20260806 22:34:08.833760 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 22230
Aug 06 22:34:08 dist-test-slave-h3ft krb5kdc[21885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.130@KRBTEST.COM
I20260806 22:34:08.857405 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:08.835114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57712 (local address 127.15.198.130:38983)
0806 22:34:08.835388 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:08.835396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:08.835426 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:34:08.835613 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:08.835619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:08.835728 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:08.835872 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:08.837780 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.838690 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.840344 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:08.840826 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:08.840984 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:08.848490 (+ 7506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:08.848525 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:08.848554 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:08.848629 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:08.853598 (+ 4969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:08.854615 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:08.854626 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:08.854632 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:08.854808 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:08.855345 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:08.855359 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:08.855363 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:08.855843 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:08.856208 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:08.856555 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 22:34:08.856836 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 22:34:08.858515 22397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:09.017657 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 22230
I20260806 22:34:09.047519 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 22415
Aug 06 22:34:09 dist-test-slave-h3ft krb5kdc[21885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.131@KRBTEST.COM
I20260806 22:34:09.068252 22679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:09.048720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34012 (local address 127.15.198.131:36643)
0806 22:34:09.049451 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:09.049457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:09.049479 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:34:09.049557 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:09.049561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:09.049645 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:09.049810 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:09.051711 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:09.052651 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:09.054073 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:09.054635 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:09.054807 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:09.060292 (+ 5485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:09.060311 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:09.060335 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:09.060389 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:09.064466 (+ 4077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:09.065387 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:09.065401 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:09.065404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:09.065576 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:09.066084 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:09.066090 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:09.066093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:09.066617 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:09.066993 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:09.067439 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 22:34:09.067722 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":401,"threads_started":1}
I20260806 22:34:09.069656 22602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:09.218225 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 22415
I20260806 22:34:09.250830 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 21904
I20260806 22:34:09.252624 21990 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:09.383982 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 21904
2026-08-06T22:34:09Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (5930 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T22:34:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:34:09Z Disabled control of system clock
I20260806 22:34:09.488045 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:42129
--webserver_interface=127.15.198.190
--webserver_port=0
--builtin_ntp_servers=127.15.198.148:38583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:42129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:34:09.916450 22688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:09.916862 22688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:09.916971 22688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:09.925335 22688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:34:09.925508 22688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:09.925554 22688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:09.925633 22688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:34:09.925683 22688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:34:09.936224 22688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:38583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:42129
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:42129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:09.938012 22688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:09.940354 22688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:09.955193 22698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:09.955631 22697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:09.955302 22702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:09.956307 22688 server_base.cc:1034] running on GCE node
I20260806 22:34:09.957444 22688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:09.959491 22688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:09.960737 22688 hybrid_clock.cc:648] HybridClock initialized: now 1786055649960686 us; error 45 us; skew 500 ppm
I20260806 22:34:09.961225 22688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:34:09.964637 22688 webserver.cc:460] Webserver started at http://127.15.198.190:42849/ using document root <none> and password file <none>
I20260806 22:34:09.965649 22688 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:09.965788 22688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:09.966253 22688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:09.970224 22688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "1049c963101b4b5099bef45378857908"
format_stamp: "Formatted at 2026-08-06 22:34:09 on dist-test-slave-h3ft"
server_key: "0696c76447de314e65456619ad739d1c"
server_key_iv: "799092240d71f970e578f8f97736530e"
server_key_version: "encryptionkey@0"
I20260806 22:34:09.971354 22688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "1049c963101b4b5099bef45378857908"
format_stamp: "Formatted at 2026-08-06 22:34:09 on dist-test-slave-h3ft"
server_key: "0696c76447de314e65456619ad739d1c"
server_key_iv: "799092240d71f970e578f8f97736530e"
server_key_version: "encryptionkey@0"
I20260806 22:34:09.978132 22688 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 22:34:09.982728 22712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:09.984733 22688 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:34:09.985028 22688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "1049c963101b4b5099bef45378857908"
format_stamp: "Formatted at 2026-08-06 22:34:09 on dist-test-slave-h3ft"
server_key: "0696c76447de314e65456619ad739d1c"
server_key_iv: "799092240d71f970e578f8f97736530e"
server_key_version: "encryptionkey@0"
I20260806 22:34:09.985365 22688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:09.993505 22688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:09.994659 22688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:09.995178 22688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:10.022873 22688 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:42129
I20260806 22:34:10.022951 22783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:42129 every 8 connection(s)
I20260806 22:34:10.024780 22688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:34:10.030843 22784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:10.034544 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 22688
I20260806 22:34:10.034873 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:34:10.035919 16154 external_mini_cluster.cc:1428] Setting key 2cbced4e6df41b644f6f4c338759b736
I20260806 22:34:10.048123 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:10.039218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55622 (local address 127.15.198.190:42129)
0806 22:34:10.040730 (+ 1512us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:10.040746 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:10.040810 (+ 64us) server_negotiation.cc:407] Connection header received
0806 22:34:10.042801 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:10.042833 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:10.043528 (+ 695us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:10.044087 (+ 559us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:10.045598 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:10.045606 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:10.045623 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:10.045644 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:10.045726 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:10.045748 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:10.046052 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:10.046221 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 22:34:10.046656 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":867,"thread_start_us":510,"threads_started":1}
I20260806 22:34:10.048749 22784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908: Bootstrap starting.
I20260806 22:34:10.054011 22784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:10.055470 22784 log.cc:824] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:10.059667 22784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908: No bootstrap required, opened a new log
I20260806 22:34:10.069463 22784 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1049c963101b4b5099bef45378857908" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42129 } }
I20260806 22:34:10.070012 22784 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:10.070093 22784 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1049c963101b4b5099bef45378857908, State: Initialized, Role: FOLLOWER
I20260806 22:34:10.070837 22784 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1049c963101b4b5099bef45378857908" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42129 } }
I20260806 22:34:10.071111 22784 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:34:10.071223 22784 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:34:10.071372 22784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:10.074599 22784 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1049c963101b4b5099bef45378857908" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42129 } }
I20260806 22:34:10.075407 22784 leader_election.cc:302] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1049c963101b4b5099bef45378857908; no voters:
I20260806 22:34:10.076162 22784 leader_election.cc:288] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:34:10.076371 22792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:10.077345 22792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [term 1 LEADER]: Becoming Leader. State: Replica: 1049c963101b4b5099bef45378857908, State: Running, Role: LEADER
I20260806 22:34:10.078115 22792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1049c963101b4b5099bef45378857908" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42129 } }
I20260806 22:34:10.079092 22784 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:34:10.082118 22793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1049c963101b4b5099bef45378857908" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1049c963101b4b5099bef45378857908" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42129 } } }
I20260806 22:34:10.082595 22793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:10.083508 22795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1049c963101b4b5099bef45378857908. Latest consensus state: current_term: 1 leader_uuid: "1049c963101b4b5099bef45378857908" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1049c963101b4b5099bef45378857908" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42129 } } }
I20260806 22:34:10.083984 22795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:10.086781 22804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:34:10.094151 22804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:34:10.107288 22804 catalog_manager.cc:1347] Generated new cluster ID: 54e067581bab4352951329f0641e1043
I20260806 22:34:10.107483 22804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:34:10.126451 22804 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:34:10.127194 22804 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:34:10.141278 22804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908: Generated new TSK 0
I20260806 22:34:10.142812 22804 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:34:10.161592 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42129
--builtin_ntp_servers=127.15.198.148:38583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:34:10.605958 22821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:10.606353 22821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:10.606469 22821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:10.614794 22821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:10.614943 22821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:10.615132 22821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:34:10.625502 22821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:38583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:10.627269 22821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:10.629607 22821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:10.646293 22827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:10.646265 22833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:10.646221 22830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:10.647713 22821 server_base.cc:1034] running on GCE node
I20260806 22:34:10.648648 22821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:10.649950 22821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:10.651196 22821 hybrid_clock.cc:648] HybridClock initialized: now 1786055650651099 us; error 89 us; skew 500 ppm
I20260806 22:34:10.651623 22821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:34:10.659221 22821 webserver.cc:460] Webserver started at http://127.15.198.129:45483/ using document root <none> and password file <none>
I20260806 22:34:10.660331 22821 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:10.660501 22821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:10.660954 22821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:10.665086 22821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "375631db14c44a5f85f0affd9a595990"
format_stamp: "Formatted at 2026-08-06 22:34:10 on dist-test-slave-h3ft"
server_key: "6d23cd5af81b7c5b220459bd786ff0f8"
server_key_iv: "ed1e69e144d88a4218dbe4d24c0247df"
server_key_version: "encryptionkey@0"
I20260806 22:34:10.666097 22821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "375631db14c44a5f85f0affd9a595990"
format_stamp: "Formatted at 2026-08-06 22:34:10 on dist-test-slave-h3ft"
server_key: "6d23cd5af81b7c5b220459bd786ff0f8"
server_key_iv: "ed1e69e144d88a4218dbe4d24c0247df"
server_key_version: "encryptionkey@0"
I20260806 22:34:10.672786 22821 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 22:34:10.677525 22843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:10.679710 22821 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:34:10.680008 22821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "375631db14c44a5f85f0affd9a595990"
format_stamp: "Formatted at 2026-08-06 22:34:10 on dist-test-slave-h3ft"
server_key: "6d23cd5af81b7c5b220459bd786ff0f8"
server_key_iv: "ed1e69e144d88a4218dbe4d24c0247df"
server_key_version: "encryptionkey@0"
I20260806 22:34:10.680368 22821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:10.688045 22821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:10.689153 22821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:10.689493 22821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:10.690858 22821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:10.692827 22821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:10.692940 22821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:10.693123 22821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:10.693213 22821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:10.737684 22821 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:42847
I20260806 22:34:10.737725 23002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:42847 every 8 connection(s)
I20260806 22:34:10.739574 22821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:34:10.747588 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 22821
I20260806 22:34:10.747895 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:34:10.748859 16154 external_mini_cluster.cc:1428] Setting key 4709e770d2315671082e73975245dad2
I20260806 22:34:10.751395 23008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:10.743250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:48809 (local address 127.15.198.190:42129)
0806 22:34:10.743903 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:10.743909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:10.745372 (+ 1463us) server_negotiation.cc:407] Connection header received
0806 22:34:10.748039 (+ 2667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:10.748047 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:10.748152 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:10.748277 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:10.750347 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:10.750354 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:10.750358 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:10.750384 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:10.750462 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:10.750478 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:10.750655 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:10.750894 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 22:34:10.750987 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":382,"threads_started":1}
I20260806 22:34:10.752789 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:10.743961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:42129 (local address 127.15.198.129:48809)
0806 22:34:10.745143 (+ 1182us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:10.745193 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:10.747450 (+ 2257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:10.748701 (+ 1251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:10.748720 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:10.748906 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:10.749878 (+ 972us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:34:10.749881 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:10.749936 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:34:10.750108 (+ 172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:10.750597 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:10.750613 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:10.750625 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 22:34:10.751045 (+ 420us) client_negotiation.cc:240] Negotiation successful
0806 22:34:10.751452 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":347,"threads_started":1}
I20260806 22:34:10.752916 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42129
--builtin_ntp_servers=127.15.198.148:38583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 22:34:10.754992 23005 heartbeater.cc:347] Connected to a master server at 127.15.198.190:42129
I20260806 22:34:10.755487 23005 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:10.756433 23005 heartbeater.cc:511] Master 127.15.198.190:42129 requested a full tablet report, sending...
I20260806 22:34:10.759327 22736 ts_manager.cc:194] Registered new tserver with Master: 375631db14c44a5f85f0affd9a595990 (127.15.198.129:42847)
W20260806 22:34:11.204558 23011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:11.204986 23011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:11.205096 23011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:11.213762 23011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:11.213999 23011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:11.214148 23011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:34:11.225359 23011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:38583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:11.227375 23011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:11.229876 23011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:11.247541 23019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:11.247509 23020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:11.247393 23024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:11.248765 23011 server_base.cc:1034] running on GCE node
I20260806 22:34:11.249776 23011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:11.251340 23011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:11.252835 23011 hybrid_clock.cc:648] HybridClock initialized: now 1786055651252680 us; error 163 us; skew 500 ppm
I20260806 22:34:11.253398 23011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:34:11.256675 23011 webserver.cc:460] Webserver started at http://127.15.198.130:34863/ using document root <none> and password file <none>
I20260806 22:34:11.257652 23011 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:11.257771 23011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:11.258268 23011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:11.262442 23011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "2e328d9db88943189eff6e4e595e6d31"
format_stamp: "Formatted at 2026-08-06 22:34:11 on dist-test-slave-h3ft"
server_key: "ca275cca2e4b24764cc4a44eb9aa84d7"
server_key_iv: "cf19fe04be7bf356fee414d76fd49a0a"
server_key_version: "encryptionkey@0"
I20260806 22:34:11.263635 23011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "2e328d9db88943189eff6e4e595e6d31"
format_stamp: "Formatted at 2026-08-06 22:34:11 on dist-test-slave-h3ft"
server_key: "ca275cca2e4b24764cc4a44eb9aa84d7"
server_key_iv: "cf19fe04be7bf356fee414d76fd49a0a"
server_key_version: "encryptionkey@0"
I20260806 22:34:11.271795 23011 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260806 22:34:11.277093 23032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:11.279276 23011 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.007s sys 0.000s
I20260806 22:34:11.279562 23011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "2e328d9db88943189eff6e4e595e6d31"
format_stamp: "Formatted at 2026-08-06 22:34:11 on dist-test-slave-h3ft"
server_key: "ca275cca2e4b24764cc4a44eb9aa84d7"
server_key_iv: "cf19fe04be7bf356fee414d76fd49a0a"
server_key_version: "encryptionkey@0"
I20260806 22:34:11.279903 23011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:11.287786 23011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:11.288981 23011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:11.289335 23011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:11.290619 23011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:11.292685 23011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:11.292824 23011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:11.292987 23011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:11.293087 23011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:11.333290 23011 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:42245
I20260806 22:34:11.333365 23211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:42245 every 8 connection(s)
I20260806 22:34:11.335193 23011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:34:11.340219 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 23011
I20260806 22:34:11.340497 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:34:11.341157 16154 external_mini_cluster.cc:1428] Setting key e00d76e004610e5c66ee8e649380aefd
I20260806 22:34:11.345379 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42129
--builtin_ntp_servers=127.15.198.148:38583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 22:34:11.346853 23214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:11.338578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:44763 (local address 127.15.198.190:42129)
0806 22:34:11.339234 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:11.339239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:11.341071 (+ 1832us) server_negotiation.cc:407] Connection header received
0806 22:34:11.343650 (+ 2579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:11.343656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:11.343737 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:11.343878 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:11.345943 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:11.345948 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:11.345952 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:11.345972 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:11.346057 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:11.346072 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:11.346202 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:11.346472 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 22:34:11.346553 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":318,"threads_started":1}
I20260806 22:34:11.348819 23215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:11.339278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:42129 (local address 127.15.198.130:44763)
0806 22:34:11.340822 (+ 1544us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:11.340883 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:11.343235 (+ 2352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:11.344296 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:11.344312 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:11.344489 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:11.345460 (+ 971us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:34:11.345464 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:11.345539 (+ 75us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:34:11.345710 (+ 171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:11.346193 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:11.346207 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:11.346220 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:34:11.346605 (+ 385us) client_negotiation.cc:240] Negotiation successful
0806 22:34:11.347096 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"thread_start_us":446,"threads_started":1}
I20260806 22:34:11.350694 23212 heartbeater.cc:347] Connected to a master server at 127.15.198.190:42129
I20260806 22:34:11.351164 23212 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:11.352169 23212 heartbeater.cc:511] Master 127.15.198.190:42129 requested a full tablet report, sending...
I20260806 22:34:11.354233 22736 ts_manager.cc:194] Registered new tserver with Master: 2e328d9db88943189eff6e4e595e6d31 (127.15.198.130:42245)
I20260806 22:34:11.763609 23005 heartbeater.cc:503] Master 127.15.198.190:42129 was elected leader, sending a full tablet report...
W20260806 22:34:11.794035 23217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:11.794461 23217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:11.794644 23217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:11.803655 23217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:11.803825 23217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:11.804032 23217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:34:11.814920 23217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:38583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:11.816851 23217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:11.819351 23217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:11.842211 23223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:11.842489 23224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:11.842726 23226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:11.844408 23217 server_base.cc:1034] running on GCE node
I20260806 22:34:11.845535 23217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:11.847075 23217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:11.848359 23217 hybrid_clock.cc:648] HybridClock initialized: now 1786055651848284 us; error 83 us; skew 500 ppm
I20260806 22:34:11.848779 23217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:34:11.851895 23217 webserver.cc:460] Webserver started at http://127.15.198.131:33945/ using document root <none> and password file <none>
I20260806 22:34:11.852869 23217 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:11.853026 23217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:11.853446 23217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:11.857234 23217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "a701d936d1b744e5898612c49e8a7c99"
format_stamp: "Formatted at 2026-08-06 22:34:11 on dist-test-slave-h3ft"
server_key: "dccbdf8f4f2f2ec11d0cd40c471cc6d5"
server_key_iv: "41243a8b9d76c4bbabc2c51d303668a4"
server_key_version: "encryptionkey@0"
I20260806 22:34:11.858240 23217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "a701d936d1b744e5898612c49e8a7c99"
format_stamp: "Formatted at 2026-08-06 22:34:11 on dist-test-slave-h3ft"
server_key: "dccbdf8f4f2f2ec11d0cd40c471cc6d5"
server_key_iv: "41243a8b9d76c4bbabc2c51d303668a4"
server_key_version: "encryptionkey@0"
I20260806 22:34:11.865106 23217 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 22:34:11.869963 23238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:11.871852 23217 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:34:11.872097 23217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "a701d936d1b744e5898612c49e8a7c99"
format_stamp: "Formatted at 2026-08-06 22:34:11 on dist-test-slave-h3ft"
server_key: "dccbdf8f4f2f2ec11d0cd40c471cc6d5"
server_key_iv: "41243a8b9d76c4bbabc2c51d303668a4"
server_key_version: "encryptionkey@0"
I20260806 22:34:11.872380 23217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:11.879845 23217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:11.881052 23217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:11.881393 23217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:11.882658 23217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:11.884712 23217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:11.884908 23217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 22:34:11.885030 23217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:11.885138 23217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:11.926421 23217 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:41589
I20260806 22:34:11.926497 23410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:41589 every 8 connection(s)
I20260806 22:34:11.928227 23217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:34:11.934002 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 23217
I20260806 22:34:11.934288 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:34:11.935086 16154 external_mini_cluster.cc:1428] Setting key f6e1f5a5650504eb3726fe266d36ecff
I20260806 22:34:11.940905 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:11.932075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:42591 (local address 127.15.198.190:42129)
0806 22:34:11.932735 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:11.932740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:11.934803 (+ 2063us) server_negotiation.cc:407] Connection header received
0806 22:34:11.937827 (+ 3024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:11.937833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:11.937938 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:11.938101 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:11.940052 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:11.940058 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:11.940062 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:11.940082 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:11.940139 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:11.940154 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:11.940269 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:11.940505 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 22:34:11.940575 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":335,"threads_started":1}
I20260806 22:34:11.942165 23415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:11.932867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:42129 (local address 127.15.198.131:42591)
0806 22:34:11.934498 (+ 1631us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:11.934575 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:11.937371 (+ 2796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:11.938452 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:11.938465 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:11.938605 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:11.939630 (+ 1025us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:34:11.939633 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:11.939686 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:34:11.939804 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:11.940255 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:11.940263 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:11.940270 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:34:11.940551 (+ 281us) client_negotiation.cc:240] Negotiation successful
0806 22:34:11.940832 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"thread_start_us":490,"threads_started":1}
I20260806 22:34:11.944056 23411 heartbeater.cc:347] Connected to a master server at 127.15.198.190:42129
I20260806 22:34:11.944986 23411 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:11.945896 23411 heartbeater.cc:511] Master 127.15.198.190:42129 requested a full tablet report, sending...
I20260806 22:34:11.948244 22736 ts_manager.cc:194] Registered new tserver with Master: a701d936d1b744e5898612c49e8a7c99 (127.15.198.131:41589)
I20260806 22:34:11.952546 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:34:11.961949 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:11.958782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55638 (local address 127.15.198.190:42129)
0806 22:34:11.959146 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:11.959152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:11.959700 (+ 548us) server_negotiation.cc:407] Connection header received
0806 22:34:11.959883 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:11.959887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:11.959959 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:11.960104 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:11.961281 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:11.961287 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:11.961290 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:11.961307 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:11.961348 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:11.961361 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:11.961494 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:11.961612 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 22:34:11.961676 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 22:34:11.967875 22736 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:34:11.976610 22736 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:34:12.002743 23425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:11.994417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:42847 (local address 127.0.0.1:41860)
0806 22:34:11.997339 (+ 2922us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:11.997384 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:11.997592 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:11.998611 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:11.998621 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:11.998753 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:11.999926 (+ 1173us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:34:11.999931 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:11.999984 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:34:12.000123 (+ 139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:12.001824 (+ 1701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:12.001831 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:12.001840 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 22:34:12.001989 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 22:34:12.002113 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2634,"spinlock_wait_cycles":10880,"thread_start_us":572,"threads_started":1}
I20260806 22:34:12.002815 23427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:11.994553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56874 (local address 127.15.198.131:41589)
0806 22:34:11.996720 (+ 2167us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:11.996737 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:11.997545 (+ 808us) server_negotiation.cc:407] Connection header received
0806 22:34:11.997829 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:11.997841 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:11.998178 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:11.998413 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:12.001204 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:12.001217 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:12.001238 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:12.001264 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:12.001356 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:12.001387 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:12.001628 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:12.002177 (+ 549us) server_negotiation.cc:299] Negotiation successful
0806 22:34:12.002330 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1872,"thread_start_us":1567,"threads_started":1}
I20260806 22:34:12.003773 23426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:11.994304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:41589 (local address 127.0.0.1:56874)
0806 22:34:11.997339 (+ 3035us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:11.997383 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:11.997594 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:12.000024 (+ 2430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:12.000029 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:12.000058 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:12.000873 (+ 815us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:34:12.000878 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:12.000901 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:34:12.000969 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:12.001699 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:12.001715 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:12.001728 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:34:12.001980 (+ 252us) client_negotiation.cc:240] Negotiation successful
0806 22:34:12.003124 (+ 1144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2694,"mutex_wait_us":1008,"spinlock_wait_cycles":24960,"thread_start_us":2290,"threads_started":1}
I20260806 22:34:12.004884 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:11.995256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41860 (local address 127.15.198.129:42847)
0806 22:34:11.998014 (+ 2758us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:11.998025 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:11.998059 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:34:11.998145 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:11.998152 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:11.998350 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:11.999484 (+ 1134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:12.001538 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:12.001546 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:12.001561 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:12.001587 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:12.001656 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:12.001684 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:12.004219 (+ 2535us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:12.004313 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 22:34:12.004451 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2539,"thread_start_us":1657,"threads_started":1}
I20260806 22:34:12.006321 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:11.997987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:42245 (local address 127.0.0.1:33420)
0806 22:34:12.001604 (+ 3617us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:12.001624 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:12.001780 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:12.002839 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:12.002843 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:12.002869 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:12.004219 (+ 1350us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:34:12.004225 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:12.004246 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:34:12.004310 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:12.004960 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:12.004966 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:12.004971 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:34:12.005663 (+ 692us) client_negotiation.cc:240] Negotiation successful
0806 22:34:12.005732 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3459,"thread_start_us":3290,"threads_started":1}
I20260806 22:34:12.007479 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:11.998034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33420 (local address 127.15.198.130:42245)
0806 22:34:12.002233 (+ 4199us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:12.002244 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:12.002283 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:34:12.002367 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:12.002377 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:12.002667 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:12.002846 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:12.004753 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:12.004759 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:12.004769 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:12.004788 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:12.004828 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:12.004842 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:12.005012 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:12.005189 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 22:34:12.007041 (+ 1852us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4014,"thread_start_us":2731,"threads_started":1}
I20260806 22:34:12.009155 23311 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7484863cef461ab758845757c1e517 (DEFAULT_TABLE table=test-table [id=8394d464f9de41389f220a2d988f01a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:12.009512 22907 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7484863cef461ab758845757c1e517 (DEFAULT_TABLE table=test-table [id=8394d464f9de41389f220a2d988f01a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:12.011346 22907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7484863cef461ab758845757c1e517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:12.011356 23311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7484863cef461ab758845757c1e517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:12.011874 23107 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7484863cef461ab758845757c1e517 (DEFAULT_TABLE table=test-table [id=8394d464f9de41389f220a2d988f01a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:12.013706 23107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7484863cef461ab758845757c1e517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:12.033761 23434 tablet_bootstrap.cc:492] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31: Bootstrap starting.
I20260806 22:34:12.037500 23433 tablet_bootstrap.cc:492] T 8e7484863cef461ab758845757c1e517 P 375631db14c44a5f85f0affd9a595990: Bootstrap starting.
I20260806 22:34:12.041599 23437 tablet_bootstrap.cc:492] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99: Bootstrap starting.
I20260806 22:34:12.043812 23434 tablet_bootstrap.cc:654] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:12.045007 23433 tablet_bootstrap.cc:654] T 8e7484863cef461ab758845757c1e517 P 375631db14c44a5f85f0affd9a595990: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:12.045403 23434 log.cc:824] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:12.046234 23437 tablet_bootstrap.cc:654] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:12.046669 23433 log.cc:824] T 8e7484863cef461ab758845757c1e517 P 375631db14c44a5f85f0affd9a595990: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:12.047820 23437 log.cc:824] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:12.054288 23433 tablet_bootstrap.cc:492] T 8e7484863cef461ab758845757c1e517 P 375631db14c44a5f85f0affd9a595990: No bootstrap required, opened a new log
I20260806 22:34:12.054286 23437 tablet_bootstrap.cc:492] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99: No bootstrap required, opened a new log
I20260806 22:34:12.054706 23437 ts_tablet_manager.cc:1397] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260806 22:34:12.054708 23433 ts_tablet_manager.cc:1397] T 8e7484863cef461ab758845757c1e517 P 375631db14c44a5f85f0affd9a595990: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.005s
I20260806 22:34:12.055102 23434 tablet_bootstrap.cc:492] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31: No bootstrap required, opened a new log
I20260806 22:34:12.055470 23434 ts_tablet_manager.cc:1397] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.006s
I20260806 22:34:12.064347 23434 raft_consensus.cc:348] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a701d936d1b744e5898612c49e8a7c99" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41589 } } peers { permanent_uuid: "375631db14c44a5f85f0affd9a595990" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 42847 } } peers { permanent_uuid: "2e328d9db88943189eff6e4e595e6d31" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 42245 } }
I20260806 22:34:12.065166 23434 raft_consensus.cc:374] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:12.065333 23434 raft_consensus.cc:729] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e328d9db88943189eff6e4e595e6d31, State: Initialized, Role: FOLLOWER
I20260806 22:34:12.066360 23434 consensus_queue.cc:260] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a701d936d1b744e5898612c49e8a7c99" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41589 } } peers { permanent_uuid: "375631db14c44a5f85f0affd9a595990" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 42847 } } peers { permanent_uuid: "2e328d9db88943189eff6e4e595e6d31" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 42245 } }
I20260806 22:34:12.067240 23437 raft_consensus.cc:348] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a701d936d1b744e5898612c49e8a7c99" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41589 } } peers { permanent_uuid: "375631db14c44a5f85f0affd9a595990" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 42847 } } peers { permanent_uuid: "2e328d9db88943189eff6e4e595e6d31" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 42245 } }
I20260806 22:34:12.067240 23433 raft_consensus.cc:348] T 8e7484863cef461ab758845757c1e517 P 375631db14c44a5f85f0affd9a595990 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a701d936d1b744e5898612c49e8a7c99" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41589 } } peers { permanent_uuid: "375631db14c44a5f85f0affd9a595990" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 42847 } } peers { permanent_uuid: "2e328d9db88943189eff6e4e595e6d31" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 42245 } }
I20260806 22:34:12.067827 23433 raft_consensus.cc:374] T 8e7484863cef461ab758845757c1e517 P 375631db14c44a5f85f0affd9a595990 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:12.067894 23437 raft_consensus.cc:374] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:12.067998 23433 raft_consensus.cc:729] T 8e7484863cef461ab758845757c1e517 P 375631db14c44a5f85f0affd9a595990 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 375631db14c44a5f85f0affd9a595990, State: Initialized, Role: FOLLOWER
I20260806 22:34:12.068084 23437 raft_consensus.cc:729] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a701d936d1b744e5898612c49e8a7c99, State: Initialized, Role: FOLLOWER
I20260806 22:34:12.068370 23212 heartbeater.cc:503] Master 127.15.198.190:42129 was elected leader, sending a full tablet report...
I20260806 22:34:12.069298 23434 ts_tablet_manager.cc:1428] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31: Time spent starting tablet: real 0.014s user 0.006s sys 0.006s
I20260806 22:34:12.069308 23437 consensus_queue.cc:260] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a701d936d1b744e5898612c49e8a7c99" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41589 } } peers { permanent_uuid: "375631db14c44a5f85f0affd9a595990" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 42847 } } peers { permanent_uuid: "2e328d9db88943189eff6e4e595e6d31" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 42245 } }
I20260806 22:34:12.069249 23433 consensus_queue.cc:260] T 8e7484863cef461ab758845757c1e517 P 375631db14c44a5f85f0affd9a595990 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a701d936d1b744e5898612c49e8a7c99" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41589 } } peers { permanent_uuid: "375631db14c44a5f85f0affd9a595990" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 42847 } } peers { permanent_uuid: "2e328d9db88943189eff6e4e595e6d31" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 42245 } }
I20260806 22:34:12.071686 23437 ts_tablet_manager.cc:1428] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99: Time spent starting tablet: real 0.017s user 0.014s sys 0.004s
I20260806 22:34:12.071935 23433 ts_tablet_manager.cc:1428] T 8e7484863cef461ab758845757c1e517 P 375631db14c44a5f85f0affd9a595990: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
I20260806 22:34:12.072718 23411 heartbeater.cc:503] Master 127.15.198.190:42129 was elected leader, sending a full tablet report...
W20260806 22:34:12.087877 23213 tablet.cc:2369] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:34:12.135102 23447 raft_consensus.cc:482] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:34:12.135550 23447 raft_consensus.cc:504] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a701d936d1b744e5898612c49e8a7c99" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41589 } } peers { permanent_uuid: "375631db14c44a5f85f0affd9a595990" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 42847 } } peers { permanent_uuid: "2e328d9db88943189eff6e4e595e6d31" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 42245 } }
I20260806 22:34:12.137709 23447 leader_election.cc:288] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 375631db14c44a5f85f0affd9a595990 (127.15.198.129:42847), 2e328d9db88943189eff6e4e595e6d31 (127.15.198.130:42245)
I20260806 22:34:12.141407 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:12.138470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:42199 (local address 127.15.198.129:42847)
0806 22:34:12.138703 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:12.138710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:12.138740 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:34:12.139140 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:12.139146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:12.139216 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:12.139385 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:12.140667 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:12.140671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:12.140675 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:12.140694 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:12.140744 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:12.140757 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:12.140936 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:12.141001 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 22:34:12.141082 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 22:34:12.141407 23415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:12.138420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:42847 (local address 127.15.198.131:42199)
0806 22:34:12.138650 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:12.138676 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:12.138822 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:12.139472 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:12.139478 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:12.139517 (+ 39us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:12.140397 (+ 880us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:34:12.140400 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:12.140426 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:34:12.140475 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:12.140872 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:12.140877 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:12.140883 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:34:12.141013 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 22:34:12.141078 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 22:34:12.141898 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:12.138503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:58323 (local address 127.15.198.130:42245)
0806 22:34:12.138703 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:12.138709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:12.139809 (+ 1100us) server_negotiation.cc:407] Connection header received
0806 22:34:12.139957 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:12.139962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:12.140026 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:12.140118 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:12.141193 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:12.141199 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:12.141204 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:12.141231 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:12.141266 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:12.141279 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:12.141417 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:12.141486 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 22:34:12.141563 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 22:34:12.142050 23449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:12.138414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:42245 (local address 127.15.198.131:58323)
0806 22:34:12.139660 (+ 1246us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:12.139675 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:12.139802 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:12.140186 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:12.140192 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:12.140221 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:12.140966 (+ 745us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:34:12.140970 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:12.140994 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:34:12.141034 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:12.141379 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:12.141384 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:12.141393 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 22:34:12.141507 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 22:34:12.141565 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1138,"mutex_wait_us":240,"thread_start_us":579,"threads_started":1}
I20260806 22:34:12.143076 22943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7484863cef461ab758845757c1e517" candidate_uuid: "a701d936d1b744e5898612c49e8a7c99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "375631db14c44a5f85f0affd9a595990" is_pre_election: true
I20260806 22:34:12.143276 23134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7484863cef461ab758845757c1e517" candidate_uuid: "a701d936d1b744e5898612c49e8a7c99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e328d9db88943189eff6e4e595e6d31" is_pre_election: true
I20260806 22:34:12.143816 22943 raft_consensus.cc:2454] T 8e7484863cef461ab758845757c1e517 P 375631db14c44a5f85f0affd9a595990 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a701d936d1b744e5898612c49e8a7c99 in term 0.
I20260806 22:34:12.143906 23134 raft_consensus.cc:2454] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a701d936d1b744e5898612c49e8a7c99 in term 0.
I20260806 22:34:12.144728 23247 leader_election.cc:302] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 375631db14c44a5f85f0affd9a595990, a701d936d1b744e5898612c49e8a7c99; no voters:
I20260806 22:34:12.145221 23447 raft_consensus.cc:2781] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:34:12.145401 23447 raft_consensus.cc:482] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:34:12.145533 23447 raft_consensus.cc:3037] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:12.148633 23447 raft_consensus.cc:504] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a701d936d1b744e5898612c49e8a7c99" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41589 } } peers { permanent_uuid: "375631db14c44a5f85f0affd9a595990" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 42847 } } peers { permanent_uuid: "2e328d9db88943189eff6e4e595e6d31" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 42245 } }
I20260806 22:34:12.149559 23447 leader_election.cc:288] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [CANDIDATE]: Term 1 election: Requested vote from peers 375631db14c44a5f85f0affd9a595990 (127.15.198.129:42847), 2e328d9db88943189eff6e4e595e6d31 (127.15.198.130:42245)
I20260806 22:34:12.150219 23134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7484863cef461ab758845757c1e517" candidate_uuid: "a701d936d1b744e5898612c49e8a7c99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e328d9db88943189eff6e4e595e6d31"
I20260806 22:34:12.150224 22943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7484863cef461ab758845757c1e517" candidate_uuid: "a701d936d1b744e5898612c49e8a7c99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "375631db14c44a5f85f0affd9a595990"
I20260806 22:34:12.150501 23134 raft_consensus.cc:3037] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:12.150511 22943 raft_consensus.cc:3037] T 8e7484863cef461ab758845757c1e517 P 375631db14c44a5f85f0affd9a595990 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:12.153275 22943 raft_consensus.cc:2454] T 8e7484863cef461ab758845757c1e517 P 375631db14c44a5f85f0affd9a595990 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a701d936d1b744e5898612c49e8a7c99 in term 1.
I20260806 22:34:12.153344 23134 raft_consensus.cc:2454] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a701d936d1b744e5898612c49e8a7c99 in term 1.
I20260806 22:34:12.153913 23247 leader_election.cc:302] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2e328d9db88943189eff6e4e595e6d31, 375631db14c44a5f85f0affd9a595990, a701d936d1b744e5898612c49e8a7c99; no voters:
I20260806 22:34:12.154347 23447 raft_consensus.cc:2781] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:12.154907 23447 raft_consensus.cc:686] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [term 1 LEADER]: Becoming Leader. State: Replica: a701d936d1b744e5898612c49e8a7c99, State: Running, Role: LEADER
I20260806 22:34:12.155699 23447 consensus_queue.cc:237] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a701d936d1b744e5898612c49e8a7c99" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41589 } } peers { permanent_uuid: "375631db14c44a5f85f0affd9a595990" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 42847 } } peers { permanent_uuid: "2e328d9db88943189eff6e4e595e6d31" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 42245 } }
I20260806 22:34:12.162854 22736 catalog_manager.cc:5495] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 reported cstate change: term changed from 0 to 1, leader changed from <none> to a701d936d1b744e5898612c49e8a7c99 (127.15.198.131). New cstate: current_term: 1 leader_uuid: "a701d936d1b744e5898612c49e8a7c99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a701d936d1b744e5898612c49e8a7c99" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "375631db14c44a5f85f0affd9a595990" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 42847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e328d9db88943189eff6e4e595e6d31" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 42245 } health_report { overall_health: UNKNOWN } } }
W20260806 22:34:12.180646 23412 tablet.cc:2369] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:34:12.210698 23427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:12.207826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56882 (local address 127.15.198.131:41589)
0806 22:34:12.208071 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:12.208076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:12.208239 (+ 163us) server_negotiation.cc:407] Connection header received
0806 22:34:12.208500 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:12.208504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:12.208619 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:12.208741 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:12.209942 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:12.209948 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:12.209952 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:12.209970 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:12.210012 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:12.210026 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:12.210159 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:12.210375 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 22:34:12.210445 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 22:34:12.222184 23134 raft_consensus.cc:1261] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31 [term 1 FOLLOWER]: Refusing update from remote peer a701d936d1b744e5898612c49e8a7c99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:12.222193 22943 raft_consensus.cc:1261] T 8e7484863cef461ab758845757c1e517 P 375631db14c44a5f85f0affd9a595990 [term 1 FOLLOWER]: Refusing update from remote peer a701d936d1b744e5898612c49e8a7c99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:12.223563 23447 consensus_queue.cc:1035] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e328d9db88943189eff6e4e595e6d31" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:12.224017 23452 consensus_queue.cc:1035] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "375631db14c44a5f85f0affd9a595990" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 42847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:12.247781 23460 mvcc.cc:205] Tried to move back new op lower bound from 7315683951488294912 to 7315683951235321856. Current Snapshot: MvccSnapshot[applied={T|T < 7315683951488294912}]
I20260806 22:34:12.265879 22735 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:34:12.266394 22735 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:34:12.272257 22735 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1049c963101b4b5099bef45378857908: Sending DeleteTablet for 3 replicas of tablet 8e7484863cef461ab758845757c1e517
I20260806 22:34:12.274150 22907 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7484863cef461ab758845757c1e517 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:12 UTC) from {username='slave'} at 127.0.0.1:41860
I20260806 22:34:12.274643 23311 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7484863cef461ab758845757c1e517 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:12 UTC) from {username='slave'} at 127.0.0.1:56874
I20260806 22:34:12.276022 23107 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7484863cef461ab758845757c1e517 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:12 UTC) from {username='slave'} at 127.0.0.1:33420
I20260806 22:34:12.279793 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 22821
I20260806 22:34:12.296056 23472 tablet_replica.cc:330] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99: stopping tablet replica
I20260806 22:34:12.296339 23473 tablet_replica.cc:330] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31: stopping tablet replica
I20260806 22:34:12.297346 23473 raft_consensus.cc:2229] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:34:12.297664 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:12.281715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41870 (local address 127.15.198.129:42847)
0806 22:34:12.282008 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:12.282019 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:12.292027 (+ 10008us) server_negotiation.cc:407] Connection header received
0806 22:34:12.292838 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:12.292849 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:12.293054 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:12.293309 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:12.296465 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:12.296473 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:12.296478 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:12.296511 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:12.296572 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:12.296593 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:12.296758 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:12.297087 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 22:34:12.297200 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 22:34:12.298283 23473 raft_consensus.cc:2258] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:12.299481 22997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:12.299566 23471 tablet_replica.cc:330] T 8e7484863cef461ab758845757c1e517 P 375631db14c44a5f85f0affd9a595990: stopping tablet replica
I20260806 22:34:12.299958 23472 raft_consensus.cc:2229] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:34:12.301366 23472 raft_consensus.cc:2258] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:12.303545 23473 ts_tablet_manager.cc:1905] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:12.304512 23472 ts_tablet_manager.cc:1905] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:12.312973 23473 ts_tablet_manager.cc:1918] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:34:12.313627 23473 log.cc:1196] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/8e7484863cef461ab758845757c1e517
I20260806 22:34:12.314404 23472 ts_tablet_manager.cc:1918] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:34:12.314594 23473 ts_tablet_manager.cc:1939] T 8e7484863cef461ab758845757c1e517 P 2e328d9db88943189eff6e4e595e6d31: Deleting consensus metadata
I20260806 22:34:12.316509 23472 log.cc:1196] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/8e7484863cef461ab758845757c1e517
I20260806 22:34:12.317621 22714 catalog_manager.cc:4841] TS 2e328d9db88943189eff6e4e595e6d31 (127.15.198.130:42245): tablet 8e7484863cef461ab758845757c1e517 (table test-table [id=8394d464f9de41389f220a2d988f01a4]) successfully deleted
I20260806 22:34:12.317929 23472 ts_tablet_manager.cc:1939] T 8e7484863cef461ab758845757c1e517 P a701d936d1b744e5898612c49e8a7c99: Deleting consensus metadata
I20260806 22:34:12.320290 22714 catalog_manager.cc:4841] TS a701d936d1b744e5898612c49e8a7c99 (127.15.198.131:41589): tablet 8e7484863cef461ab758845757c1e517 (table test-table [id=8394d464f9de41389f220a2d988f01a4]) successfully deleted
I20260806 22:34:12.300016 23471 raft_consensus.cc:2229] T 8e7484863cef461ab758845757c1e517 P 375631db14c44a5f85f0affd9a595990 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:34:12.473893 23471 raft_consensus.cc:2258] T 8e7484863cef461ab758845757c1e517 P 375631db14c44a5f85f0affd9a595990 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:12.474231 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 22821
W20260806 22:34:12.512084 22720 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.198.129:42847 (error 108)
I20260806 22:34:12.512482 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 23011
I20260806 22:34:12.515913 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:12.514029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:42847 (local address 127.0.0.1:41874)
0806 22:34:12.515475 (+ 1446us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:12.515639 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1292,"thread_start_us":996,"threads_started":1}
I20260806 22:34:12.516548 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:12.513648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33426 (local address 127.15.198.130:42245)
0806 22:34:12.513892 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:12.513899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:12.513928 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:34:12.514084 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:12.514091 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:12.514266 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:12.514435 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:12.515767 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:12.515772 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:12.515776 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:12.515797 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:12.515839 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:12.515854 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:12.516025 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:12.516118 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 22:34:12.516201 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 22:34:12.516559 22720 catalog_manager.cc:4584] TS 375631db14c44a5f85f0affd9a595990 (127.15.198.129:42847): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8e7484863cef461ab758845757c1e517: Network error: Client connection negotiation failed: client connection to 127.15.198.129:42847: connect: Connection refused (error 111)
I20260806 22:34:12.517619 23205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:12.570925 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:12.570253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:42847 (local address 127.0.0.1:41888)
0806 22:34:12.570549 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:12.570684 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 22:34:12.572459 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:12.571907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:42847 (local address 127.0.0.1:41902)
0806 22:34:12.572104 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:12.572236 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 22:34:12.621930 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 23011
I20260806 22:34:12.634852 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:12.634100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:42847 (local address 127.0.0.1:41906)
0806 22:34:12.634486 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:12.634622 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211}
I20260806 22:34:12.636683 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:12.636071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:42847 (local address 127.0.0.1:41912)
0806 22:34:12.636336 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:12.636461 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260806 22:34:12.660387 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 23217
I20260806 22:34:12.664803 23427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:12.661749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56888 (local address 127.15.198.131:41589)
0806 22:34:12.662002 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:12.662011 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:12.662130 (+ 119us) server_negotiation.cc:407] Connection header received
0806 22:34:12.662430 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:12.662435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:12.662507 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:12.662641 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:12.663968 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:12.663974 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:12.663977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:12.663997 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:12.664045 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:12.664061 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:12.664249 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:12.664346 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 22:34:12.664425 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260806 22:34:12.665782 23405 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:12.717937 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:12.717306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:42847 (local address 127.0.0.1:41916)
0806 22:34:12.717629 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:12.717741 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260806 22:34:12.719471 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:12.718874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:42847 (local address 127.0.0.1:41922)
0806 22:34:12.719183 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:12.719267 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205}
I20260806 22:34:12.807098 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 23217
I20260806 22:34:12.840032 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 22688
I20260806 22:34:12.841741 22778 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:12.973344 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:12.972021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:42847 (local address 127.0.0.1:41936)
0806 22:34:12.972928 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:12.973072 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439}
I20260806 22:34:12.973902 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 22688
2026-08-06T22:34:12Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3588 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 22:34:13.030171 16154 mini_oidc.cc:152] Starting JWKS server
I20260806 22:34:13.033762 16154 webserver.cc:460] Webserver started at https://127.0.0.1:38867/ using document root <none> and password file <none>
I20260806 22:34:13.034106 16154 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:34:13.035173 16154 webserver.cc:460] Webserver started at http://127.0.0.1:43845/ using document root <none> and password file <none>
2026-08-06T22:34:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:34:13Z Disabled control of system clock
I20260806 22:34:13.085400 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:33587
--webserver_interface=127.15.198.190
--webserver_port=0
--builtin_ntp_servers=127.15.198.148:45103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:33587
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38867/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/testchainca.cert with env {}
W20260806 22:34:13.523149 23490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:13.523625 23490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:13.523802 23490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:13.532243 23490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:34:13.532457 23490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:13.532585 23490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:13.532702 23490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:34:13.532785 23490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:34:13.532886 23490 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:34:13.532972 23490 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38867/jwks.json
I20260806 22:34:13.544819 23490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:45103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:33587
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:33587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38867/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:13.546723 23490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:13.548601 23490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:13.566150 23498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:13.566172 23500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:13.566622 23497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:13.567566 23490 server_base.cc:1034] running on GCE node
I20260806 22:34:13.568725 23490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:13.570786 23490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:13.572068 23490 hybrid_clock.cc:648] HybridClock initialized: now 1786055653571972 us; error 78 us; skew 500 ppm
I20260806 22:34:13.572576 23490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:34:13.575632 23490 webserver.cc:460] Webserver started at http://127.15.198.190:33329/ using document root <none> and password file <none>
I20260806 22:34:13.576573 23490 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:13.576735 23490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:13.577131 23490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:13.580916 23490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "d00d7f0146134fd4878001547914d387"
format_stamp: "Formatted at 2026-08-06 22:34:13 on dist-test-slave-h3ft"
server_key: "ca05e1cfaf353ee2b6c3b7656bd3d017"
server_key_iv: "a7fffc2a3362997f921ad49d0d667099"
server_key_version: "encryptionkey@0"
I20260806 22:34:13.581897 23490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "d00d7f0146134fd4878001547914d387"
format_stamp: "Formatted at 2026-08-06 22:34:13 on dist-test-slave-h3ft"
server_key: "ca05e1cfaf353ee2b6c3b7656bd3d017"
server_key_iv: "a7fffc2a3362997f921ad49d0d667099"
server_key_version: "encryptionkey@0"
I20260806 22:34:13.589035 23490 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260806 22:34:13.593962 23510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:13.596062 23490 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:34:13.596331 23490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "d00d7f0146134fd4878001547914d387"
format_stamp: "Formatted at 2026-08-06 22:34:13 on dist-test-slave-h3ft"
server_key: "ca05e1cfaf353ee2b6c3b7656bd3d017"
server_key_iv: "a7fffc2a3362997f921ad49d0d667099"
server_key_version: "encryptionkey@0"
I20260806 22:34:13.596657 23490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:13.623440 23490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:13.624676 23490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:13.625072 23490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:13.647975 23490 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:33587
I20260806 22:34:13.647997 23585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:33587 every 8 connection(s)
I20260806 22:34:13.649878 23490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:34:13.652784 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 23490
I20260806 22:34:13.653079 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:34:13.654057 16154 external_mini_cluster.cc:1428] Setting key e02fcbe5851f14c89ce99d4f41f9fa3d
I20260806 22:34:13.656196 23586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:13.670921 23586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387: Bootstrap starting.
I20260806 22:34:13.671241 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:13.656763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60314 (local address 127.15.198.190:33587)
0806 22:34:13.658014 (+ 1251us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:13.658029 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:13.658088 (+ 59us) server_negotiation.cc:407] Connection header received
0806 22:34:13.660058 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:13.660088 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:13.660709 (+ 621us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:13.661231 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:13.661996 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:13.663698 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:13.665395 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:13.666191 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:13.666425 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:13.668074 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:13.668084 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:13.668108 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:13.668881 (+ 773us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:13.668980 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:13.669021 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:13.669170 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:13.669460 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 22:34:13.669891 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":434,"threads_started":1}
I20260806 22:34:13.675887 23586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:13.676544 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:34:13.677378 23586 log.cc:824] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:13.681457 23586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387: No bootstrap required, opened a new log
I20260806 22:34:13.690677 23586 raft_consensus.cc:348] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00d7f0146134fd4878001547914d387" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33587 } }
I20260806 22:34:13.691381 23586 raft_consensus.cc:374] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:13.691512 23586 raft_consensus.cc:729] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d00d7f0146134fd4878001547914d387, State: Initialized, Role: FOLLOWER
I20260806 22:34:13.692332 23586 consensus_queue.cc:260] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00d7f0146134fd4878001547914d387" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33587 } }
I20260806 22:34:13.692667 23586 raft_consensus.cc:388] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:34:13.692848 23586 raft_consensus.cc:482] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:34:13.693068 23586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:13.696077 23586 raft_consensus.cc:504] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00d7f0146134fd4878001547914d387" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33587 } }
I20260806 22:34:13.696799 23586 leader_election.cc:302] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d00d7f0146134fd4878001547914d387; no voters:
I20260806 22:34:13.697566 23586 leader_election.cc:288] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:34:13.697767 23593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:13.698578 23593 raft_consensus.cc:686] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [term 1 LEADER]: Becoming Leader. State: Replica: d00d7f0146134fd4878001547914d387, State: Running, Role: LEADER
I20260806 22:34:13.699410 23593 consensus_queue.cc:237] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00d7f0146134fd4878001547914d387" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33587 } }
I20260806 22:34:13.700522 23586 sys_catalog.cc:564] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:34:13.703150 23594 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d00d7f0146134fd4878001547914d387" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00d7f0146134fd4878001547914d387" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33587 } } }
I20260806 22:34:13.704190 23594 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:13.703358 23595 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d00d7f0146134fd4878001547914d387. Latest consensus state: current_term: 1 leader_uuid: "d00d7f0146134fd4878001547914d387" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00d7f0146134fd4878001547914d387" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33587 } } }
I20260806 22:34:13.707469 23595 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:13.711664 23603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:34:13.718660 23603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:34:13.730144 23603 catalog_manager.cc:1347] Generated new cluster ID: 4d3287e376404b64ae09693c269a8516
I20260806 22:34:13.730335 23603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:34:13.748329 23603 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:34:13.750851 23603 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:34:13.761246 23603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d00d7f0146134fd4878001547914d387: Generated new TSK 0
I20260806 22:34:13.762812 23603 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 22:34:13.830232 23628 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 22:34:13.830654 23628 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 22:34:13.831751 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:13.798504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60318 (local address 127.15.198.190:33587)
0806 22:34:13.798838 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:13.798852 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:13.799546 (+ 694us) server_negotiation.cc:407] Connection header received
0806 22:34:13.800127 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:13.800133 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:13.800248 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:13.800440 (+ 192us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:34:13.801364 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:13.802403 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:13.804901 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:13.805458 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:13.805594 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:13.806523 (+ 929us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:34:13.830674 (+ 24151us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38867/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 22:34:13.831340 (+ 666us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60318: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38867/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":95}
W20260806 22:34:13.832130 23589 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60318: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38867/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 22:34:13.834672 23627 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:13.798386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33587 (local address 127.0.0.1:60318)
0806 22:34:13.799350 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:13.799376 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:13.799644 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:13.800519 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:13.800524 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:13.800539 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:34:13.801066 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:13.801083 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:13.802582 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:13.802589 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:13.804630 (+ 2041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:13.804656 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:13.805680 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:13.805688 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:13.805915 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:13.806097 (+ 182us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:34:13.831210 (+ 25113us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38867/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 22:34:13.831743 (+ 533us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.198.190:33587: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38867/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":499,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.15.198.190:33587: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38867/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 22:34:13.836992 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 23490
I20260806 22:34:13.839043 23581 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:13.953119 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 23490
2026-08-06T22:34:14Z chronyd exiting
I20260806 22:34:14.021054 16154 test_util.cc:175] -----------------------------------------------
I20260806 22:34:14.021243 16154 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055595907839-16154-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (996 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 22:34:14.025920 16154 mini_oidc.cc:152] Starting JWKS server
I20260806 22:34:14.028280 16154 webserver.cc:460] Webserver started at https://127.0.0.1:39253/ using document root <none> and password file <none>
I20260806 22:34:14.028599 16154 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:34:14.029390 16154 webserver.cc:460] Webserver started at http://127.0.0.1:40921/ using document root <none> and password file <none>
2026-08-06T22:34:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:34:14Z Disabled control of system clock
I20260806 22:34:14.079697 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:45845
--webserver_interface=127.15.198.190
--webserver_port=0
--builtin_ntp_servers=127.15.198.148:33821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:45845
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39253/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/testchainca.cert with env {}
W20260806 22:34:14.526108 23640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:14.526610 23640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:14.526798 23640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:14.536679 23640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:34:14.536895 23640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:14.537000 23640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:14.537134 23640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:34:14.537220 23640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:34:14.537333 23640 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:34:14.537406 23640 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39253/jwks.json
I20260806 22:34:14.549780 23640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:33821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:45845
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:45845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39253/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:14.551931 23640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:14.553774 23640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:14.569442 23646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:14.569630 23648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:14.569635 23645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:14.570739 23640 server_base.cc:1034] running on GCE node
I20260806 22:34:14.571949 23640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:14.574069 23640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:14.575523 23640 hybrid_clock.cc:648] HybridClock initialized: now 1786055654575393 us; error 62 us; skew 500 ppm
I20260806 22:34:14.576212 23640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:34:14.579394 23640 webserver.cc:460] Webserver started at http://127.15.198.190:46535/ using document root <none> and password file <none>
I20260806 22:34:14.580399 23640 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:14.580571 23640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:14.580989 23640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:14.584935 23640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "0dcf1ba1e157445091c22ae78153ec70"
format_stamp: "Formatted at 2026-08-06 22:34:14 on dist-test-slave-h3ft"
server_key: "e36afbe6f9f8ac540526c4482d2d414c"
server_key_iv: "7f710a7f5ac35410a035c8731784bcf5"
server_key_version: "encryptionkey@0"
I20260806 22:34:14.585955 23640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "0dcf1ba1e157445091c22ae78153ec70"
format_stamp: "Formatted at 2026-08-06 22:34:14 on dist-test-slave-h3ft"
server_key: "e36afbe6f9f8ac540526c4482d2d414c"
server_key_iv: "7f710a7f5ac35410a035c8731784bcf5"
server_key_version: "encryptionkey@0"
I20260806 22:34:14.593091 23640 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260806 22:34:14.598068 23660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:14.600414 23640 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:34:14.600711 23640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "0dcf1ba1e157445091c22ae78153ec70"
format_stamp: "Formatted at 2026-08-06 22:34:14 on dist-test-slave-h3ft"
server_key: "e36afbe6f9f8ac540526c4482d2d414c"
server_key_iv: "7f710a7f5ac35410a035c8731784bcf5"
server_key_version: "encryptionkey@0"
I20260806 22:34:14.601035 23640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:14.629628 23640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:14.630905 23640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:14.631353 23640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:14.654745 23640 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:45845
I20260806 22:34:14.654768 23741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:45845 every 8 connection(s)
I20260806 22:34:14.656605 23640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:34:14.658190 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 23640
I20260806 22:34:14.658494 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:34:14.659494 16154 external_mini_cluster.cc:1428] Setting key c940d1ccd3d2867e2f0cee6207076b66
I20260806 22:34:14.664222 23743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:14.676560 23748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:14.661829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37586 (local address 127.15.198.190:45845)
0806 22:34:14.663505 (+ 1676us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:14.663521 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:14.663578 (+ 57us) server_negotiation.cc:407] Connection header received
0806 22:34:14.665730 (+ 2152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:14.665760 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:14.666337 (+ 577us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:14.666746 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:14.667645 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:14.669166 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:14.671173 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:14.671933 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:14.672099 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:14.673573 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:14.673581 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:14.673598 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:14.674272 (+ 674us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:14.674364 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:14.674388 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:14.674502 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:14.674766 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 22:34:14.675240 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":827,"thread_start_us":499,"threads_started":1}
I20260806 22:34:14.679140 23743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70: Bootstrap starting.
I20260806 22:34:14.681591 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:34:14.683478 23743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:14.684901 23743 log.cc:824] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:14.689080 23743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70: No bootstrap required, opened a new log
I20260806 22:34:14.697849 23743 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dcf1ba1e157445091c22ae78153ec70" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 45845 } }
I20260806 22:34:14.698362 23743 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:14.698504 23743 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dcf1ba1e157445091c22ae78153ec70, State: Initialized, Role: FOLLOWER
I20260806 22:34:14.699340 23743 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dcf1ba1e157445091c22ae78153ec70" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 45845 } }
I20260806 22:34:14.699604 23743 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:34:14.699817 23743 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:34:14.700172 23743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:14.703456 23743 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dcf1ba1e157445091c22ae78153ec70" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 45845 } }
I20260806 22:34:14.704306 23743 leader_election.cc:302] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dcf1ba1e157445091c22ae78153ec70; no voters:
I20260806 22:34:14.705111 23743 leader_election.cc:288] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:34:14.705277 23754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:14.706195 23754 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [term 1 LEADER]: Becoming Leader. State: Replica: 0dcf1ba1e157445091c22ae78153ec70, State: Running, Role: LEADER
I20260806 22:34:14.706841 23754 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dcf1ba1e157445091c22ae78153ec70" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 45845 } }
I20260806 22:34:14.707988 23743 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:34:14.710271 23756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dcf1ba1e157445091c22ae78153ec70. Latest consensus state: current_term: 1 leader_uuid: "0dcf1ba1e157445091c22ae78153ec70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dcf1ba1e157445091c22ae78153ec70" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 45845 } } }
I20260806 22:34:14.710709 23756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:14.711501 23755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0dcf1ba1e157445091c22ae78153ec70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dcf1ba1e157445091c22ae78153ec70" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 45845 } } }
I20260806 22:34:14.712786 23755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:14.714011 23765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:34:14.720484 23765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:34:14.732589 23765 catalog_manager.cc:1347] Generated new cluster ID: 57aaf95d63ec4fb099f6c6744a016a6b
I20260806 22:34:14.732807 23765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:34:14.768805 23765 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:34:14.770515 23765 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:34:14.779059 23765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0dcf1ba1e157445091c22ae78153ec70: Generated new TSK 0
I20260806 22:34:14.780227 23765 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 22:34:14.822615 23795 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 22:34:14.822801 23795 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 22:34:14.824505 23748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:14.801915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37588 (local address 127.15.198.190:45845)
0806 22:34:14.802174 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:14.802182 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:14.802683 (+ 501us) server_negotiation.cc:407] Connection header received
0806 22:34:14.803076 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:14.803086 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:14.803191 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:14.803408 (+ 217us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:34:14.803950 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:14.804797 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:14.806556 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:14.807233 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:14.807393 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:14.808044 (+ 651us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:34:14.823532 (+ 15488us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39253/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 22:34:14.824093 (+ 561us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37588: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39253/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 22:34:14.824679 23748 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37588: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39253/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 22:34:14.824756 23794 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:14.801742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:45845 (local address 127.0.0.1:37588)
0806 22:34:14.802547 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:14.802567 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:14.802781 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:14.803428 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:14.803433 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:14.803447 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:34:14.803774 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:14.803783 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:14.804984 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:14.804989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:14.806390 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:14.806411 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:14.807475 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:14.807484 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:14.807655 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:14.807699 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:34:14.823891 (+ 16192us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39253/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 22:34:14.824250 (+ 359us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.198.190:45845: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39253/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":457,"threads_started":1}
I20260806 22:34:14.826818 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 23640
I20260806 22:34:14.828425 23734 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:14.943838 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 23640
2026-08-06T22:34:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (989 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 22:34:15.015905 16154 mini_oidc.cc:152] Starting JWKS server
I20260806 22:34:15.018599 16154 webserver.cc:460] Webserver started at https://127.0.0.1:45089/ using document root <none> and password file <none>
I20260806 22:34:15.018965 16154 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:34:15.019960 16154 webserver.cc:460] Webserver started at http://127.0.0.1:44769/ using document root <none> and password file <none>
2026-08-06T22:34:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:34:15Z Disabled control of system clock
I20260806 22:34:15.071888 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:37799
--webserver_interface=127.15.198.190
--webserver_port=0
--builtin_ntp_servers=127.15.198.148:39515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:37799
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45089/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 22:34:15.533347 23804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:15.533759 23804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:15.533979 23804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:15.542333 23804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:34:15.542498 23804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:15.542601 23804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:15.542721 23804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:34:15.542809 23804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:34:15.542935 23804 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:34:15.543042 23804 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45089/jwks.json
I20260806 22:34:15.553984 23804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:37799
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:37799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45089/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:15.556008 23804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:15.558494 23804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:15.573575 23812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:15.573483 23813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:15.573552 23815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:15.574698 23804 server_base.cc:1034] running on GCE node
I20260806 22:34:15.575852 23804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:15.577839 23804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:15.579160 23804 hybrid_clock.cc:648] HybridClock initialized: now 1786055655579099 us; error 50 us; skew 500 ppm
I20260806 22:34:15.579617 23804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:34:15.582751 23804 webserver.cc:460] Webserver started at http://127.15.198.190:36655/ using document root <none> and password file <none>
I20260806 22:34:15.583864 23804 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:15.584044 23804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:15.584489 23804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:15.588406 23804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "7993629763e242879f9eaf20f5a4ef05"
format_stamp: "Formatted at 2026-08-06 22:34:15 on dist-test-slave-h3ft"
server_key: "6e92d011c3911ae5e4787da9b6bb7a1c"
server_key_iv: "052bc119aefd1b0974130bc1fde59bdb"
server_key_version: "encryptionkey@0"
I20260806 22:34:15.589480 23804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "7993629763e242879f9eaf20f5a4ef05"
format_stamp: "Formatted at 2026-08-06 22:34:15 on dist-test-slave-h3ft"
server_key: "6e92d011c3911ae5e4787da9b6bb7a1c"
server_key_iv: "052bc119aefd1b0974130bc1fde59bdb"
server_key_version: "encryptionkey@0"
I20260806 22:34:15.596355 23804 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260806 22:34:15.601392 23822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:15.603765 23804 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 22:34:15.604100 23804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "7993629763e242879f9eaf20f5a4ef05"
format_stamp: "Formatted at 2026-08-06 22:34:15 on dist-test-slave-h3ft"
server_key: "6e92d011c3911ae5e4787da9b6bb7a1c"
server_key_iv: "052bc119aefd1b0974130bc1fde59bdb"
server_key_version: "encryptionkey@0"
I20260806 22:34:15.604447 23804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:15.622530 23804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:15.623744 23804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:15.624150 23804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:15.648139 23804 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:37799
I20260806 22:34:15.648159 23905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:37799 every 8 connection(s)
I20260806 22:34:15.650154 23804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:34:15.650767 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 23804
I20260806 22:34:15.651118 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:34:15.652146 16154 external_mini_cluster.cc:1428] Setting key 44b8fa3be9bb30cfce5257839c915036
I20260806 22:34:15.656785 23907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:15.673552 23907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05: Bootstrap starting.
I20260806 22:34:15.679384 23907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:15.679600 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:15.654850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49562 (local address 127.15.198.190:37799)
0806 22:34:15.656330 (+ 1480us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:15.656350 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:15.656416 (+ 66us) server_negotiation.cc:407] Connection header received
0806 22:34:15.659186 (+ 2770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:15.659236 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:15.660065 (+ 829us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:15.660670 (+ 605us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:15.661564 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:15.663894 (+ 2330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:15.665858 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:15.667240 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:15.667490 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:15.675166 (+ 7676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:15.675181 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:15.675206 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:15.676293 (+ 1087us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:15.676455 (+ 162us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:15.676496 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:15.676664 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:15.677065 (+ 401us) server_negotiation.cc:299] Negotiation successful
0806 22:34:15.677734 (+ 669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798,"thread_start_us":448,"threads_started":1}
I20260806 22:34:15.680989 23907 log.cc:824] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:15.685559 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:34:15.686017 23907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05: No bootstrap required, opened a new log
I20260806 22:34:15.698345 23907 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7993629763e242879f9eaf20f5a4ef05" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 37799 } }
I20260806 22:34:15.698984 23907 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:15.699203 23907 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7993629763e242879f9eaf20f5a4ef05, State: Initialized, Role: FOLLOWER
I20260806 22:34:15.700054 23907 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7993629763e242879f9eaf20f5a4ef05" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 37799 } }
I20260806 22:34:15.700371 23907 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:34:15.700541 23907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:34:15.700775 23907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:15.704107 23907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7993629763e242879f9eaf20f5a4ef05" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 37799 } }
I20260806 22:34:15.704872 23907 leader_election.cc:302] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7993629763e242879f9eaf20f5a4ef05; no voters:
I20260806 22:34:15.705648 23907 leader_election.cc:288] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:34:15.705860 23917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:15.706740 23917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [term 1 LEADER]: Becoming Leader. State: Replica: 7993629763e242879f9eaf20f5a4ef05, State: Running, Role: LEADER
I20260806 22:34:15.707500 23917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7993629763e242879f9eaf20f5a4ef05" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 37799 } }
I20260806 22:34:15.708487 23907 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:34:15.712255 23918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7993629763e242879f9eaf20f5a4ef05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7993629763e242879f9eaf20f5a4ef05" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 37799 } } }
I20260806 22:34:15.712839 23918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:15.713411 23919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7993629763e242879f9eaf20f5a4ef05. Latest consensus state: current_term: 1 leader_uuid: "7993629763e242879f9eaf20f5a4ef05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7993629763e242879f9eaf20f5a4ef05" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 37799 } } }
I20260806 22:34:15.713824 23919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:15.716814 23930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:34:15.724184 23930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:34:15.739193 23930 catalog_manager.cc:1347] Generated new cluster ID: ae69a019cc0642d8bea3a3f4ff61abe4
I20260806 22:34:15.739403 23930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:34:15.762485 23930 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:34:15.764294 23930 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:34:15.772025 23930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7993629763e242879f9eaf20f5a4ef05: Generated new TSK 0
I20260806 22:34:15.773304 23930 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:34:15.803612 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:15.791858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49578 (local address 127.15.198.190:37799)
0806 22:34:15.792141 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:15.792149 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:15.792613 (+ 464us) server_negotiation.cc:407] Connection header received
0806 22:34:15.792928 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:15.792936 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:15.793061 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:15.793306 (+ 245us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:34:15.794418 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:15.795847 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:15.797540 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:15.798556 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:15.798730 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:15.799662 (+ 932us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:34:15.802724 (+ 3062us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45089/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 22:34:15.803209 (+ 485us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49578: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45089/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":97}
W20260806 22:34:15.803777 23911 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49578: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45089/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 22:34:15.805564 23954 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:15.791685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:37799 (local address 127.0.0.1:49578)
0806 22:34:15.792434 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:15.792464 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:15.792656 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:15.793484 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:15.793491 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:15.793523 (+ 32us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:34:15.794170 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:15.794187 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:15.796086 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:15.796092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:15.797329 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:15.797347 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:15.798850 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:15.798856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:15.799186 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:15.799246 (+ 60us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:34:15.803071 (+ 3825us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45089/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 22:34:15.805008 (+ 1937us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.198.190:37799: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45089/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":1455,"thread_start_us":399,"threads_started":1}
I20260806 22:34:15.806871 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 23804
I20260806 22:34:15.808526 23897 generic_service.cc:196] Checking for leaks (request via RPC)
E20260806 22:34:15.816879 23955 webserver.cc:533] Webserver: sslize failed:
E20260806 22:34:15.817034 23955 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 22:34:15.914449 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 23804
2026-08-06T22:34:15Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (968 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T22:34:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:34:16Z Disabled control of system clock
I20260806 22:34:16.032337 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:41435
--webserver_interface=127.15.198.190
--webserver_port=0
--builtin_ntp_servers=127.15.198.148:37655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:41435
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/testchainca.cert with env {}
W20260806 22:34:16.475302 23961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:16.475723 23961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:16.475826 23961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:16.484151 23961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:34:16.484308 23961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:16.484350 23961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:16.484406 23961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:34:16.484455 23961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:34:16.484524 23961 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:34:16.484567 23961 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 22:34:16.497793 23961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:37655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:41435
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:41435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:16.500365 23961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:16.502290 23961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:16.519364 23969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:16.519465 23967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:16.519419 23972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:16.520727 23961 server_base.cc:1034] running on GCE node
I20260806 22:34:16.521940 23961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:16.524199 23961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:16.525489 23961 hybrid_clock.cc:648] HybridClock initialized: now 1786055656525440 us; error 46 us; skew 500 ppm
I20260806 22:34:16.526010 23961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:34:16.529567 23961 webserver.cc:460] Webserver started at http://127.15.198.190:33713/ using document root <none> and password file <none>
I20260806 22:34:16.530771 23961 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:16.530905 23961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:16.531492 23961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:16.535679 23961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "d3a1a99bbf1948e29aea62c4409dad99"
format_stamp: "Formatted at 2026-08-06 22:34:16 on dist-test-slave-h3ft"
server_key: "451854ba1b383be3a07e6dadfa3d4c6c"
server_key_iv: "381e6dd00a4bb2723f2f58daf5509779"
server_key_version: "encryptionkey@0"
I20260806 22:34:16.536782 23961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "d3a1a99bbf1948e29aea62c4409dad99"
format_stamp: "Formatted at 2026-08-06 22:34:16 on dist-test-slave-h3ft"
server_key: "451854ba1b383be3a07e6dadfa3d4c6c"
server_key_iv: "381e6dd00a4bb2723f2f58daf5509779"
server_key_version: "encryptionkey@0"
I20260806 22:34:16.543934 23961 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 22:34:16.548810 23980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:16.551152 23961 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:34:16.551416 23961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "d3a1a99bbf1948e29aea62c4409dad99"
format_stamp: "Formatted at 2026-08-06 22:34:16 on dist-test-slave-h3ft"
server_key: "451854ba1b383be3a07e6dadfa3d4c6c"
server_key_iv: "381e6dd00a4bb2723f2f58daf5509779"
server_key_version: "encryptionkey@0"
I20260806 22:34:16.551719 23961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:16.583223 23961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:16.584430 23961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:16.584820 23961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:16.608562 23961 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:41435
I20260806 22:34:16.608582 24049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:41435 every 8 connection(s)
I20260806 22:34:16.610467 23961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:34:16.616099 24050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:16.621084 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 23961
I20260806 22:34:16.621380 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:34:16.622349 16154 external_mini_cluster.cc:1428] Setting key 6f327e90311211c98a544787d0176646
I20260806 22:34:16.630555 24050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99: Bootstrap starting.
I20260806 22:34:16.634593 24050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:16.636106 24050 log.cc:824] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:16.639604 24058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:16.624952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56742 (local address 127.15.198.190:41435)
0806 22:34:16.626179 (+ 1227us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:16.626194 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:16.626251 (+ 57us) server_negotiation.cc:407] Connection header received
0806 22:34:16.628958 (+ 2707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:16.628992 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:16.629594 (+ 602us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:16.630074 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:16.630793 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:16.632306 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:16.633830 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:16.634615 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:16.634870 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:16.636397 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:16.636408 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:16.636425 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:34:16.637367 (+ 942us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:16.637513 (+ 146us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:34:16.637546 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:16.637682 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:16.637954 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 22:34:16.638346 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":394,"threads_started":1}
I20260806 22:34:16.640065 24050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99: No bootstrap required, opened a new log
I20260806 22:34:16.644378 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:34:16.649461 24050 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a1a99bbf1948e29aea62c4409dad99" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41435 } }
I20260806 22:34:16.650040 24050 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:16.650170 24050 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3a1a99bbf1948e29aea62c4409dad99, State: Initialized, Role: FOLLOWER
I20260806 22:34:16.650895 24050 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a1a99bbf1948e29aea62c4409dad99" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41435 } }
I20260806 22:34:16.651265 24050 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:34:16.651458 24050 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:34:16.651700 24050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:16.654275 24050 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a1a99bbf1948e29aea62c4409dad99" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41435 } }
I20260806 22:34:16.654963 24050 leader_election.cc:302] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3a1a99bbf1948e29aea62c4409dad99; no voters:
I20260806 22:34:16.655840 24050 leader_election.cc:288] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:34:16.656031 24060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:16.657076 24060 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [term 1 LEADER]: Becoming Leader. State: Replica: d3a1a99bbf1948e29aea62c4409dad99, State: Running, Role: LEADER
I20260806 22:34:16.657869 24060 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a1a99bbf1948e29aea62c4409dad99" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41435 } }
I20260806 22:34:16.658739 24050 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:34:16.661207 24061 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3a1a99bbf1948e29aea62c4409dad99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a1a99bbf1948e29aea62c4409dad99" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41435 } } }
I20260806 22:34:16.661639 24061 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:16.662915 24062 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3a1a99bbf1948e29aea62c4409dad99. Latest consensus state: current_term: 1 leader_uuid: "d3a1a99bbf1948e29aea62c4409dad99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a1a99bbf1948e29aea62c4409dad99" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41435 } } }
I20260806 22:34:16.663414 24062 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:16.664227 24066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:34:16.671325 24066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:34:16.682976 24066 catalog_manager.cc:1347] Generated new cluster ID: cce85d98f6884adeb5d50052cf435975
I20260806 22:34:16.683168 24066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:34:16.700486 24066 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:34:16.702301 24066 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:34:16.710294 24066 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3a1a99bbf1948e29aea62c4409dad99: Generated new TSK 0
I20260806 22:34:16.711920 24066 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:34:17.619851 24058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:16.740708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56748 (local address 127.15.198.190:41435)
0806 22:34:16.741010 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:16.741021 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:16.741560 (+ 539us) server_negotiation.cc:407] Connection header received
0806 22:34:16.741849 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:16.741853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:16.741947 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:16.742127 (+ 180us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:34:16.742687 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:16.743645 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:16.745810 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:16.746363 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:16.746504 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:16.747260 (+ 756us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:34:17.618859 (+871599us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 22:34:17.619477 (+ 618us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56748: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":87}
W20260806 22:34:17.620002 24058 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56748: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 22:34:17.620066 24092 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:16.740667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41435 (local address 127.0.0.1:56748)
0806 22:34:16.741413 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:16.741437 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:16.741635 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:16.742166 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:16.742172 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:16.742186 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:34:16.742518 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:16.742532 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:16.743801 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:16.743808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:16.745638 (+ 1830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:16.745654 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:16.746577 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:16.746584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:16.746779 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:16.746842 (+ 63us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:34:17.619304 (+872462us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 22:34:17.619661 (+ 357us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.198.190:41435: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":388,"threads_started":1}
I20260806 22:34:17.621769 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 23961
I20260806 22:34:17.623217 24044 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:17.745209 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 23961
2026-08-06T22:34:17Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1814 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (537 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (533 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:34:18 dist-test-slave-h3ft krb5kdc[24113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:34:18 dist-test-slave-h3ft krb5kdc[24113](info): set up 2 sockets
Aug 06 22:34:18 dist-test-slave-h3ft krb5kdc[24113](info): commencing operation
krb5kdc: starting...
W20260806 22:34:20.182448 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:34:20 dist-test-slave-h3ft krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:34:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:34:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:34:20.531865 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:36523
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:36523
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:20.963591 24132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:20.964075 24132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:20.964259 24132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:20.974205 24132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:34:20.974409 24132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:20.974555 24132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:20.974677 24132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:34:20.974777 24132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:34:20.986786 24132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:36523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:36523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:20.988749 24132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:20.990432 24132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:21.005656 24137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:21.005592 24138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:21.005565 24140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:21.006812 24132 server_base.cc:1034] running on GCE node
I20260806 22:34:21.008128 24132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:21.010022 24132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:21.011332 24132 hybrid_clock.cc:648] HybridClock initialized: now 1786055661011274 us; error 47 us; skew 500 ppm
I20260806 22:34:21.011777 24132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:21 dist-test-slave-h3ft krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055661, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:21.020741 24132 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:34:21.022960 24132 webserver.cc:460] Webserver started at http://127.15.198.190:44203/ using document root <none> and password file <none>
I20260806 22:34:21.024014 24132 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:21.024196 24132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:21.024641 24132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:21.028641 24132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "1ca68b55969a403faf96be934a25a1c6"
format_stamp: "Formatted at 2026-08-06 22:34:21 on dist-test-slave-h3ft"
server_key: "ef456b52d07a8f6b727982bb1fea3c4b"
server_key_iv: "0efc8d40a441168060df71f33f1b453e"
server_key_version: "encryptionkey@0"
I20260806 22:34:21.029598 24132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "1ca68b55969a403faf96be934a25a1c6"
format_stamp: "Formatted at 2026-08-06 22:34:21 on dist-test-slave-h3ft"
server_key: "ef456b52d07a8f6b727982bb1fea3c4b"
server_key_iv: "0efc8d40a441168060df71f33f1b453e"
server_key_version: "encryptionkey@0"
I20260806 22:34:21.036412 24132 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 22:34:21.041011 24149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:21.042946 24132 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260806 22:34:21.043246 24132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "1ca68b55969a403faf96be934a25a1c6"
format_stamp: "Formatted at 2026-08-06 22:34:21 on dist-test-slave-h3ft"
server_key: "ef456b52d07a8f6b727982bb1fea3c4b"
server_key_iv: "0efc8d40a441168060df71f33f1b453e"
server_key_version: "encryptionkey@0"
I20260806 22:34:21.043521 24132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:21.066418 24132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:21.073796 24132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:21.074246 24132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:21.099526 24132 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:36523
I20260806 22:34:21.099551 24223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:36523 every 8 connection(s)
I20260806 22:34:21.101405 24132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:34:21.106635 24224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:21.108462 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 24132
I20260806 22:34:21.108767 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:34:21.109730 16154 external_mini_cluster.cc:1428] Setting key c56f4178fa50a5415853a89135c01661
I20260806 22:34:21.124967 24224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6: Bootstrap starting.
Aug 06 22:34:21 dist-test-slave-h3ft krb5kdc[24113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:34:21.130024 24224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:21.131685 24224 log.cc:824] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:21.135650 24224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6: No bootstrap required, opened a new log
I20260806 22:34:21.138948 24228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:21.112546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57198 (local address 127.15.198.190:36523)
0806 22:34:21.113585 (+ 1039us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:21.113600 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:21.113654 (+ 54us) server_negotiation.cc:407] Connection header received
0806 22:34:21.115706 (+ 2052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:21.115744 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:21.116319 (+ 575us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:21.116750 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:21.119070 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:21.120705 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:21.122735 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:21.123643 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:21.123808 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:21.129636 (+ 5828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:21.129648 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:21.129670 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:21.129732 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:21.133799 (+ 4067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:21.134740 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:21.134748 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:21.134756 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:21.134963 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:21.135478 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:21.135485 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:21.135493 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:21.136219 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:21.136605 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:21.137129 (+ 524us) server_negotiation.cc:299] Negotiation successful
0806 22:34:21.137652 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":459,"threads_started":1}
I20260806 22:34:21.145639 24224 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca68b55969a403faf96be934a25a1c6" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36523 } }
I20260806 22:34:21.146327 24224 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:21.146461 24224 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca68b55969a403faf96be934a25a1c6, State: Initialized, Role: FOLLOWER
I20260806 22:34:21.147346 24224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca68b55969a403faf96be934a25a1c6" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36523 } }
I20260806 22:34:21.147648 24224 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:34:21.147830 24224 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:34:21.148066 24224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:21.151412 24224 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca68b55969a403faf96be934a25a1c6" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36523 } }
I20260806 22:34:21.152150 24224 leader_election.cc:302] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ca68b55969a403faf96be934a25a1c6; no voters:
I20260806 22:34:21.152921 24224 leader_election.cc:288] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:34:21.153174 24231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:21.153970 24231 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [term 1 LEADER]: Becoming Leader. State: Replica: 1ca68b55969a403faf96be934a25a1c6, State: Running, Role: LEADER
I20260806 22:34:21.154603 24231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca68b55969a403faf96be934a25a1c6" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36523 } }
I20260806 22:34:21.155876 24224 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:34:21.158349 24234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ca68b55969a403faf96be934a25a1c6. Latest consensus state: current_term: 1 leader_uuid: "1ca68b55969a403faf96be934a25a1c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca68b55969a403faf96be934a25a1c6" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36523 } } }
I20260806 22:34:21.159195 24234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:21.158983 24232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ca68b55969a403faf96be934a25a1c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca68b55969a403faf96be934a25a1c6" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36523 } } }
I20260806 22:34:21.159371 24232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:21.161319 24243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:34:21.169569 24243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:34:21.182039 24243 catalog_manager.cc:1347] Generated new cluster ID: 3622f0c401b54f0192bf1bf6c8526863
I20260806 22:34:21.182221 24243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:34:21.203534 24243 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:34:21.205960 24243 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:34:21.217047 24243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ca68b55969a403faf96be934a25a1c6: Generated new TSK 0
I20260806 22:34:21.218305 24243 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
I20260806 22:34:21.304392 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36523
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:21.739188 24264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:21.739638 24264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:21.739861 24264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:21.748713 24264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:21.748953 24264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:21.749212 24264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:34:21.761577 24264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:21.763684 24264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:21.765553 24264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:21.783447 24270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:21.783378 24272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:21.783481 24269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:21.784246 24264 server_base.cc:1034] running on GCE node
I20260806 22:34:21.785739 24264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:21.787200 24264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:21.788487 24264 hybrid_clock.cc:648] HybridClock initialized: now 1786055661788423 us; error 82 us; skew 500 ppm
I20260806 22:34:21.788949 24264 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:21 dist-test-slave-h3ft krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055661, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:21.793396 24264 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:34:21.795661 24264 webserver.cc:460] Webserver started at http://127.15.198.129:32979/ using document root <none> and password file <none>
I20260806 22:34:21.796710 24264 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:21.796902 24264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:21.797348 24264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:21.801450 24264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "a059b644d87e4168b55d3bac63c43ef5"
format_stamp: "Formatted at 2026-08-06 22:34:21 on dist-test-slave-h3ft"
server_key: "f085283e53f7e1b4e7a5421bdc9d4603"
server_key_iv: "c4910831c561905bc82fc41cab8d0b5a"
server_key_version: "encryptionkey@0"
I20260806 22:34:21.802526 24264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "a059b644d87e4168b55d3bac63c43ef5"
format_stamp: "Formatted at 2026-08-06 22:34:21 on dist-test-slave-h3ft"
server_key: "f085283e53f7e1b4e7a5421bdc9d4603"
server_key_iv: "c4910831c561905bc82fc41cab8d0b5a"
server_key_version: "encryptionkey@0"
I20260806 22:34:21.809540 24264 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 22:34:21.814476 24287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:21.816337 24264 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:34:21.816570 24264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "a059b644d87e4168b55d3bac63c43ef5"
format_stamp: "Formatted at 2026-08-06 22:34:21 on dist-test-slave-h3ft"
server_key: "f085283e53f7e1b4e7a5421bdc9d4603"
server_key_iv: "c4910831c561905bc82fc41cab8d0b5a"
server_key_version: "encryptionkey@0"
I20260806 22:34:21.816860 24264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:21.833331 24264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:21.839406 24264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:21.839800 24264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:21.841207 24264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:21.843273 24264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:21.843405 24264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:21.843595 24264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:21.843684 24264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:21.887187 24264 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:34313
I20260806 22:34:21.887241 24449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:34313 every 8 connection(s)
I20260806 22:34:21.889045 24264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:34:21.893139 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 24264
I20260806 22:34:21.893571 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:34:21.894614 16154 external_mini_cluster.cc:1428] Setting key daaf021479ddcb9ecd8f6831f6b76c29
Aug 06 22:34:21 dist-test-slave-h3ft krb5kdc[24113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055661, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:34:21.917711 24452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:21.892500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:60923 (local address 127.15.198.190:36523)
0806 22:34:21.893368 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:21.893379 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:21.894702 (+ 1323us) server_negotiation.cc:407] Connection header received
0806 22:34:21.897808 (+ 3106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:21.897815 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:21.897930 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:21.898080 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:21.900363 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:21.901630 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:21.903733 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:21.904498 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:21.904659 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:21.909690 (+ 5031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:21.909699 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:21.909707 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:21.909766 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:21.913169 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:21.914437 (+ 1268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:21.914443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:21.914445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:21.914639 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:21.915398 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:21.915402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:21.915405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:21.915779 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:21.915949 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:21.917156 (+ 1207us) server_negotiation.cc:299] Negotiation successful
0806 22:34:21.917407 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":342,"threads_started":1}
I20260806 22:34:21.919782 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:21.893277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:36523 (local address 127.15.198.129:60923)
0806 22:34:21.894474 (+ 1197us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:21.894525 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:21.897296 (+ 2771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:21.898402 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:21.898414 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:21.899266 (+ 852us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:21.900128 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:21.900154 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:21.901868 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:21.901874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:21.903553 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:21.903570 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:21.904796 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:21.904803 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:21.905102 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:21.906178 (+ 1076us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:21.906211 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:21.909351 (+ 3140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:21.913373 (+ 4022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:21.913388 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:21.913417 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:21.914244 (+ 827us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:21.914825 (+ 581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:21.914833 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:21.914839 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:21.915209 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:21.915947 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:21.915960 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:21.916618 (+ 658us) client_negotiation.cc:769] Sending connection context
0806 22:34:21.917007 (+ 389us) client_negotiation.cc:240] Negotiation successful
0806 22:34:21.917835 (+ 828us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"thread_start_us":470,"threads_started":1}
I20260806 22:34:21.921976 24450 heartbeater.cc:347] Connected to a master server at 127.15.198.190:36523
I20260806 22:34:21.922806 24450 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:21.923916 24450 heartbeater.cc:511] Master 127.15.198.190:36523 requested a full tablet report, sending...
I20260806 22:34:21.927202 24174 ts_manager.cc:194] Registered new tserver with Master: a059b644d87e4168b55d3bac63c43ef5 (127.15.198.129:34313)
I20260806 22:34:21.929275 24174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:60923
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
I20260806 22:34:21.972604 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36523
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:22.407774 24456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:22.408234 24456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:22.408429 24456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:22.416980 24456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:22.417207 24456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:22.417421 24456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:34:22.429239 24456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:22.431210 24456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:22.432920 24456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:22.449980 24461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:22.449770 24464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:22.449747 24462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:22.451256 24456 server_base.cc:1034] running on GCE node
I20260806 22:34:22.452144 24456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:22.453446 24456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:22.454733 24456 hybrid_clock.cc:648] HybridClock initialized: now 1786055662454653 us; error 75 us; skew 500 ppm
I20260806 22:34:22.455241 24456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:22 dist-test-slave-h3ft krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055662, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:22.459628 24456 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:34:22.461799 24456 webserver.cc:460] Webserver started at http://127.15.198.130:34187/ using document root <none> and password file <none>
I20260806 22:34:22.462877 24456 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:22.463078 24456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:22.463507 24456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:22.467343 24456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "2584be23b1fb4d49bacfb232ed745bfe"
format_stamp: "Formatted at 2026-08-06 22:34:22 on dist-test-slave-h3ft"
server_key: "2b9104439f80b1639527637c3f34ca28"
server_key_iv: "ca42336d9b123c88cbff28373e6bb3ab"
server_key_version: "encryptionkey@0"
I20260806 22:34:22.468345 24456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "2584be23b1fb4d49bacfb232ed745bfe"
format_stamp: "Formatted at 2026-08-06 22:34:22 on dist-test-slave-h3ft"
server_key: "2b9104439f80b1639527637c3f34ca28"
server_key_iv: "ca42336d9b123c88cbff28373e6bb3ab"
server_key_version: "encryptionkey@0"
I20260806 22:34:22.475088 24456 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 22:34:22.479696 24479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:22.481577 24456 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260806 22:34:22.481827 24456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "2584be23b1fb4d49bacfb232ed745bfe"
format_stamp: "Formatted at 2026-08-06 22:34:22 on dist-test-slave-h3ft"
server_key: "2b9104439f80b1639527637c3f34ca28"
server_key_iv: "ca42336d9b123c88cbff28373e6bb3ab"
server_key_version: "encryptionkey@0"
I20260806 22:34:22.482125 24456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:22.504493 24456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:22.510116 24456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:22.510509 24456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:22.511848 24456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:22.513764 24456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:22.513887 24456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:22.514073 24456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:22.514158 24456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:22.554937 24456 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:38383
I20260806 22:34:22.554975 24637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:38383 every 8 connection(s)
I20260806 22:34:22.557004 24456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:34:22.560060 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 24456
I20260806 22:34:22.560305 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:34:22.560863 16154 external_mini_cluster.cc:1428] Setting key 01bb2e69b5aa9b49bf0d4956151ee002
Aug 06 22:34:22 dist-test-slave-h3ft krb5kdc[24113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055662, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
I20260806 22:34:22.601179 24642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:22.560595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:59197 (local address 127.15.198.190:36523)
0806 22:34:22.567629 (+ 7034us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:22.567640 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:22.576165 (+ 8525us) server_negotiation.cc:407] Connection header received
0806 22:34:22.579768 (+ 3603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:22.579774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:22.579891 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:22.580119 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:22.582522 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:22.583507 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:22.585639 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:22.586489 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:22.586937 (+ 448us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:22.592470 (+ 5533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:22.592477 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:22.592482 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:22.592535 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:22.596270 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:22.597708 (+ 1438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:22.597715 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:22.597718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:22.597911 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:22.598485 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:22.598489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:22.598499 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:22.598961 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:22.599229 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:22.600373 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0806 22:34:22.600718 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6830,"thread_start_us":6670,"threads_started":1}
I20260806 22:34:22.602593 24643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:22.561682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:36523 (local address 127.15.198.130:59197)
0806 22:34:22.575712 (+ 14030us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:22.575812 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:22.579277 (+ 3465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:22.580535 (+ 1258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:22.580549 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:22.581345 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:22.582225 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:22.582250 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:22.583752 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:22.583756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:22.585372 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:22.585391 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:22.586939 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:22.586949 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:22.587457 (+ 508us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:22.588684 (+ 1227us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:22.588727 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:22.592032 (+ 3305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:22.596548 (+ 4516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:22.596559 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:22.596584 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:22.597504 (+ 920us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:22.598060 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:22.598064 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:22.598066 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:22.598342 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:22.599195 (+ 853us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:22.599204 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:22.599857 (+ 653us) client_negotiation.cc:769] Sending connection context
0806 22:34:22.600279 (+ 422us) client_negotiation.cc:240] Negotiation successful
0806 22:34:22.601068 (+ 789us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13479,"thread_start_us":14347,"threads_started":1}
I20260806 22:34:22.605443 24638 heartbeater.cc:347] Connected to a master server at 127.15.198.190:36523
I20260806 22:34:22.605981 24638 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:22.607141 24638 heartbeater.cc:511] Master 127.15.198.190:36523 requested a full tablet report, sending...
I20260806 22:34:22.609434 24174 ts_manager.cc:194] Registered new tserver with Master: 2584be23b1fb4d49bacfb232ed745bfe (127.15.198.130:38383)
I20260806 22:34:22.611214 24174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:59197
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
I20260806 22:34:22.643055 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36523
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:34:22.934065 24450 heartbeater.cc:503] Master 127.15.198.190:36523 was elected leader, sending a full tablet report...
W20260806 22:34:23.082545 24647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:23.083299 24647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:23.083503 24647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:23.092396 24647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:23.092599 24647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:23.092828 24647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:34:23.105031 24647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:23.107041 24647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:23.108995 24647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:23.127350 24655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:23.127583 24654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:23.127338 24658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:23.128881 24647 server_base.cc:1034] running on GCE node
I20260806 22:34:23.129827 24647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:23.131321 24647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:23.132616 24647 hybrid_clock.cc:648] HybridClock initialized: now 1786055663132551 us; error 54 us; skew 500 ppm
I20260806 22:34:23.133090 24647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:23 dist-test-slave-h3ft krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055663, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:23.137607 24647 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:34:23.139954 24647 webserver.cc:460] Webserver started at http://127.15.198.131:43219/ using document root <none> and password file <none>
I20260806 22:34:23.140969 24647 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:23.141170 24647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:23.141606 24647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:23.146586 24647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "517bc12734644ec8b247f3ada983bbfd"
format_stamp: "Formatted at 2026-08-06 22:34:23 on dist-test-slave-h3ft"
server_key: "0b0095bce90675616255f057b1111503"
server_key_iv: "f6f47ac8e7f06c80337cfec9a183320f"
server_key_version: "encryptionkey@0"
I20260806 22:34:23.147874 24647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "517bc12734644ec8b247f3ada983bbfd"
format_stamp: "Formatted at 2026-08-06 22:34:23 on dist-test-slave-h3ft"
server_key: "0b0095bce90675616255f057b1111503"
server_key_iv: "f6f47ac8e7f06c80337cfec9a183320f"
server_key_version: "encryptionkey@0"
I20260806 22:34:23.155486 24647 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 22:34:23.160696 24672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:23.162837 24647 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260806 22:34:23.163168 24647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "517bc12734644ec8b247f3ada983bbfd"
format_stamp: "Formatted at 2026-08-06 22:34:23 on dist-test-slave-h3ft"
server_key: "0b0095bce90675616255f057b1111503"
server_key_iv: "f6f47ac8e7f06c80337cfec9a183320f"
server_key_version: "encryptionkey@0"
I20260806 22:34:23.163503 24647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:23.178666 24647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:23.185161 24647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:23.185585 24647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:23.187083 24647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:23.188827 24647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:23.188947 24647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:23.189118 24647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:23.189196 24647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:23.232374 24647 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:46257
I20260806 22:34:23.232424 24845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:46257 every 8 connection(s)
I20260806 22:34:23.234197 24647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:34:23.241291 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 24647
I20260806 22:34:23.241573 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:34:23.242236 16154 external_mini_cluster.cc:1428] Setting key 212abf96c32c5f4b487fda7d9b3b3f29
Aug 06 22:34:23 dist-test-slave-h3ft krb5kdc[24113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055663, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:34:23.263216 24849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:23.237657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:37041 (local address 127.15.198.190:36523)
0806 22:34:23.238268 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:23.238273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:23.239320 (+ 1047us) server_negotiation.cc:407] Connection header received
0806 22:34:23.241825 (+ 2505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:23.241831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:23.241980 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:23.242102 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:23.244567 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:23.245682 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:23.248029 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:23.248682 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:23.248829 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:23.254130 (+ 5301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:23.254137 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:23.254141 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:23.254189 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:23.258257 (+ 4068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:23.259704 (+ 1447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:23.259710 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:23.259750 (+ 40us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:23.259975 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:23.260573 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:23.260577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:23.260580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:23.260981 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:23.261146 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:23.262327 (+ 1181us) server_negotiation.cc:299] Negotiation successful
0806 22:34:23.262703 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":331,"threads_started":1}
I20260806 22:34:23.264524 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:23.238151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:36523 (local address 127.15.198.131:37041)
0806 22:34:23.239122 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:23.239168 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:23.241384 (+ 2216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:23.242468 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:23.242483 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:23.243372 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:23.244325 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:23.244360 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:23.245895 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:23.245902 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:23.247807 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:23.247828 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:23.249074 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:23.249090 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:23.249417 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:23.250490 (+ 1073us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:23.250521 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:23.253712 (+ 3191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:23.258592 (+ 4880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:23.258610 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:23.258644 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:23.259519 (+ 875us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:23.260144 (+ 625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:23.260150 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:23.260154 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:23.260444 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:23.261162 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:23.261170 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:23.261684 (+ 514us) client_negotiation.cc:769] Sending connection context
0806 22:34:23.262166 (+ 482us) client_negotiation.cc:240] Negotiation successful
0806 22:34:23.262809 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":343,"threads_started":1}
I20260806 22:34:23.266680 24846 heartbeater.cc:347] Connected to a master server at 127.15.198.190:36523
I20260806 22:34:23.267160 24846 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:23.268304 24846 heartbeater.cc:511] Master 127.15.198.190:36523 requested a full tablet report, sending...
I20260806 22:34:23.270635 24174 ts_manager.cc:194] Registered new tserver with Master: 517bc12734644ec8b247f3ada983bbfd (127.15.198.131:46257)
I20260806 22:34:23.272112 24174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:37041
I20260806 22:34:23.273379 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/test-admin.keytab.
Aug 06 22:34:23 dist-test-slave-h3ft krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:23.315086 16154 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 22:34:23.321934 24857 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:23.318622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35114 (local address 127.15.198.129:34313)
0806 22:34:23.319504 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:23.319514 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:23.319550 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:34:23.319632 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:23.319639 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:23.319918 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:23.320081 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:23.321333 (+ 1252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35114: BlockingRecv error: recv got EOF from 127.0.0.1:35114 (error 108)
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":464,"threads_started":1}
W20260806 22:34:23.322803 16154 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/krb5cc)
Aug 06 22:34:23 dist-test-slave-h3ft krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:23.324874 16154 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:34:23 dist-test-slave-h3ft krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:23.326730 16154 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:34:23 dist-test-slave-h3ft krb5kdc[24113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:34:23.347334 24857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:23.329381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35126 (local address 127.15.198.129:34313)
0806 22:34:23.329707 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:23.329718 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:23.330077 (+ 359us) server_negotiation.cc:407] Connection header received
0806 22:34:23.330364 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:23.330369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:23.330452 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:23.330579 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:23.331992 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:23.333136 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:23.334466 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:23.335291 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:23.335466 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:23.340302 (+ 4836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:23.340321 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:23.340342 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:23.340392 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:23.343846 (+ 3454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:23.344652 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:23.344662 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:23.344665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:23.344817 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:23.345253 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:23.345259 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:23.345269 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:23.345747 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:23.346223 (+ 476us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:23.346591 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 22:34:23.346914 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
W20260806 22:34:23.355262 24857 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:23.352973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35128 (local address 127.15.198.129:34313)
0806 22:34:23.353265 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:23.353272 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:23.353536 (+ 264us) server_negotiation.cc:407] Connection header received
0806 22:34:23.353766 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:23.353771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:23.353867 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:23.353984 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:23.354888 (+ 904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35128: BlockingRecv error: recv got EOF from 127.0.0.1:35128 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260806 22:34:23.355527 16154 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/krb5cc' of type FILE
Aug 06 22:34:23 dist-test-slave-h3ft krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:23.357555 16154 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:34:23 dist-test-slave-h3ft krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:23.359233 16154 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:34:23 dist-test-slave-h3ft krb5kdc[24113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:34:23.377166 24857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:23.361039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35144 (local address 127.15.198.129:34313)
0806 22:34:23.361310 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:23.361320 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:23.361617 (+ 297us) server_negotiation.cc:407] Connection header received
0806 22:34:23.361889 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:23.361894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:23.361979 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:23.362089 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:23.363309 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:23.364126 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:23.365422 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:23.365886 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:23.366062 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:23.370766 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:23.370772 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:23.370776 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:23.370833 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:23.374012 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:23.374855 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:23.374860 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:23.374862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:23.375154 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:23.375634 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:23.375638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:23.375640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:23.376012 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:23.376252 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:23.376634 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 22:34:23.376846 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260806 22:34:23.384145 24857 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:23.381579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35152 (local address 127.15.198.129:34313)
0806 22:34:23.381867 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:23.381874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:23.382200 (+ 326us) server_negotiation.cc:407] Connection header received
0806 22:34:23.382485 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:23.382489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:23.382579 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:23.382699 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:23.383837 (+ 1138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35152: BlockingRecv error: recv got EOF from 127.0.0.1:35152 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
W20260806 22:34:23.384419 16154 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055595907839-16154-0/krb5kdc/krb5cc)
Aug 06 22:34:23 dist-test-slave-h3ft krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:23.386545 16154 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:34:23 dist-test-slave-h3ft krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:23.388577 16154 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:34:23 dist-test-slave-h3ft krb5kdc[24113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:34:23.410082 24857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:23.390592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35158 (local address 127.15.198.129:34313)
0806 22:34:23.390897 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:23.390912 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:23.391198 (+ 286us) server_negotiation.cc:407] Connection header received
0806 22:34:23.391403 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:23.391408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:23.391492 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:23.391592 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:23.392956 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:23.393818 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:23.395389 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:23.395935 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:23.396070 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:23.402489 (+ 6419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:23.402496 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:23.402500 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:23.402676 (+ 176us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:23.406438 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:23.407694 (+ 1256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:23.407701 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:23.407704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:23.407956 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:23.408390 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:23.408394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:23.408396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:23.408753 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:23.409095 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:23.409382 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 22:34:23.409695 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 22:34:23.412371 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 24264
I20260806 22:34:23.427832 24857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:23.413449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35170 (local address 127.15.198.129:34313)
0806 22:34:23.413761 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:23.413770 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:23.414141 (+ 371us) server_negotiation.cc:407] Connection header received
0806 22:34:23.414385 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:23.414392 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:23.414480 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:23.414597 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:23.415821 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:23.416600 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:23.417966 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:23.418502 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:23.418658 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:23.421406 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:23.421412 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:23.421416 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:23.421455 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:23.424718 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:23.425544 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:23.425550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:23.425553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:23.425715 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:23.426152 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:23.426156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:23.426159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:23.426492 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:23.426809 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:23.427193 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 22:34:23.427457 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 22:34:23.428678 24444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:23.525652 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 24264
I20260806 22:34:23.556391 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 24456
Aug 06 22:34:23 dist-test-slave-h3ft krb5kdc[24113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.130@KRBTEST.COM
I20260806 22:34:23.577906 24873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:23.557661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34144 (local address 127.15.198.130:38383)
0806 22:34:23.558598 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:23.558610 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:23.558650 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:34:23.558737 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:23.558748 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:23.559108 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:23.559306 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:23.560567 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:23.562290 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:23.564012 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:23.564681 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:23.564812 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:23.570142 (+ 5330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:23.570170 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:23.570193 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:23.570251 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:23.573938 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:23.575120 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:23.575136 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:23.575142 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:23.575370 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:23.575875 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:23.575879 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:23.575881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:23.576407 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:23.576699 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:23.577131 (+ 432us) server_negotiation.cc:299] Negotiation successful
0806 22:34:23.577486 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":509,"threads_started":1}
I20260806 22:34:23.579869 24631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:23.676247 24638 heartbeater.cc:503] Master 127.15.198.190:36523 was elected leader, sending a full tablet report...
I20260806 22:34:23.678558 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 24456
I20260806 22:34:23.706683 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 24647
Aug 06 22:34:23 dist-test-slave-h3ft krb5kdc[24113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.131@KRBTEST.COM
I20260806 22:34:23.727268 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:23.708166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53920 (local address 127.15.198.131:46257)
0806 22:34:23.708852 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:23.708861 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:23.708901 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:34:23.708984 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:23.708991 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:23.709289 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:23.709529 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:23.710857 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:23.712267 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:23.713741 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:23.714448 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:23.714636 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:23.719897 (+ 5261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:23.719918 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:23.719944 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:23.720007 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:23.723778 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:23.724721 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:23.724730 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:23.724733 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:23.724916 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:23.725348 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:23.725353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:23.725355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:23.725814 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:23.726136 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:23.726491 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 22:34:23.726756 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":343,"threads_started":1}
I20260806 22:34:23.729146 24840 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:23.824455 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 24647
I20260806 22:34:23.853572 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 24132
I20260806 22:34:23.855335 24218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:23.980973 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 24132
2026-08-06T22:34:24Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (5172 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:34:24 dist-test-slave-h3ft krb5kdc[24884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:34:24 dist-test-slave-h3ft krb5kdc[24884](info): set up 2 sockets
Aug 06 22:34:24 dist-test-slave-h3ft krb5kdc[24884](info): commencing operation
krb5kdc: starting...
W20260806 22:34:25.381477 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.339s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:34:25 dist-test-slave-h3ft krb5kdc[24884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:34:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:34:25Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.190.keytab.
Entry for principal oryx/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.190.keytab.
I20260806 22:34:25.758252 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:38077
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.190.keytab
--principal=oryx/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:32981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:38077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:26.199980 24905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:26.200585 24905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:26.200896 24905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:26.209868 24905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:34:26.210053 24905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:26.210150 24905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:26.210248 24905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:34:26.210331 24905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:34:26.222466 24905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:32981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:38077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.190.keytab
--principal=oryx/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:38077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:26.224545 24905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:26.226357 24905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:26.243162 24912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:26.243162 24916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:26.243434 24914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:26.244275 24905 server_base.cc:1034] running on GCE node
I20260806 22:34:26.245731 24905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:26.248117 24905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:26.249439 24905 hybrid_clock.cc:648] HybridClock initialized: now 1786055666249373 us; error 61 us; skew 500 ppm
I20260806 22:34:26.249984 24905 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:26 dist-test-slave-h3ft krb5kdc[24884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055666, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:26.260253 24905 init.cc:377] Logged in from keytab as oryx/127.15.198.190@KRBTEST.COM (short username oryx)
I20260806 22:34:26.262425 24905 webserver.cc:460] Webserver started at http://127.15.198.190:44817/ using document root <none> and password file <none>
I20260806 22:34:26.263557 24905 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:26.263710 24905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:26.264079 24905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:26.268301 24905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "27fab5d8c68046e3973078cbdc8d5841"
format_stamp: "Formatted at 2026-08-06 22:34:26 on dist-test-slave-h3ft"
server_key: "595a793449f9b9a6507b51c83dab4b7d"
server_key_iv: "35001f7ab9ec5a8d76c9bbafa2795c1e"
server_key_version: "encryptionkey@0"
I20260806 22:34:26.269316 24905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "27fab5d8c68046e3973078cbdc8d5841"
format_stamp: "Formatted at 2026-08-06 22:34:26 on dist-test-slave-h3ft"
server_key: "595a793449f9b9a6507b51c83dab4b7d"
server_key_iv: "35001f7ab9ec5a8d76c9bbafa2795c1e"
server_key_version: "encryptionkey@0"
I20260806 22:34:26.276876 24905 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 22:34:26.282366 24927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:26.284744 24905 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:34:26.285081 24905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "27fab5d8c68046e3973078cbdc8d5841"
format_stamp: "Formatted at 2026-08-06 22:34:26 on dist-test-slave-h3ft"
server_key: "595a793449f9b9a6507b51c83dab4b7d"
server_key_iv: "35001f7ab9ec5a8d76c9bbafa2795c1e"
server_key_version: "encryptionkey@0"
I20260806 22:34:26.285341 24905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:26.311359 24905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:26.317932 24905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:26.318356 24905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:26.341988 24905 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:38077
I20260806 22:34:26.342022 25004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:38077 every 8 connection(s)
I20260806 22:34:26.344014 24905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:34:26.347144 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 24905
I20260806 22:34:26.347447 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:34:26.348217 16154 external_mini_cluster.cc:1428] Setting key 7370531e63d3938c7a517be217816157
I20260806 22:34:26.349572 25005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 22:34:26 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM
I20260806 22:34:26.366592 25005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841: Bootstrap starting.
I20260806 22:34:26.372257 25005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:26.374044 25005 log.cc:824] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:26.378243 25005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841: No bootstrap required, opened a new log
I20260806 22:34:26.378355 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:26.350881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53700 (local address 127.15.198.190:38077)
0806 22:34:26.351959 (+ 1078us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:26.351977 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:26.352044 (+ 67us) server_negotiation.cc:407] Connection header received
0806 22:34:26.354189 (+ 2145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:26.354229 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:26.354938 (+ 709us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:26.355571 (+ 633us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:26.357283 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:26.359544 (+ 2261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:26.360968 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:26.362082 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:26.362275 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:26.367407 (+ 5132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:26.367420 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:26.367448 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:26.367519 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:26.372277 (+ 4758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:26.373461 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:26.373471 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:26.373484 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:26.373788 (+ 304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:26.374430 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:26.374436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:26.374440 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:26.375376 (+ 936us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:26.375819 (+ 443us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:26.376398 (+ 579us) server_negotiation.cc:299] Negotiation successful
0806 22:34:26.377023 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":402,"threads_started":1}
I20260806 22:34:26.388016 25005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } }
I20260806 22:34:26.388628 25005 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:26.388757 25005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27fab5d8c68046e3973078cbdc8d5841, State: Initialized, Role: FOLLOWER
I20260806 22:34:26.389698 25005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } }
I20260806 22:34:26.390053 25005 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:34:26.390218 25005 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:34:26.390440 25005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:26.393296 25005 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } }
I20260806 22:34:26.393946 25005 leader_election.cc:302] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27fab5d8c68046e3973078cbdc8d5841; no voters:
I20260806 22:34:26.394713 25005 leader_election.cc:288] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:34:26.394918 25010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:26.396503 25010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [term 1 LEADER]: Becoming Leader. State: Replica: 27fab5d8c68046e3973078cbdc8d5841, State: Running, Role: LEADER
I20260806 22:34:26.397315 25010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } }
I20260806 22:34:26.397605 25005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:34:26.401348 25012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27fab5d8c68046e3973078cbdc8d5841. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } }
I20260806 22:34:26.401993 25012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:26.401372 25011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } }
I20260806 22:34:26.402750 25011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:26.403028 25021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:34:26.409987 25021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:34:26.423182 25021 catalog_manager.cc:1347] Generated new cluster ID: c13aa0eae9f240ceb2a6470f0e8d05bb
I20260806 22:34:26.423353 25021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:34:26.454612 25021 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:34:26.456530 25021 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:34:26.467208 25021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841: Generated new TSK 0
I20260806 22:34:26.468791 25021 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.129.keytab.
Entry for principal oryx/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.129.keytab.
I20260806 22:34:26.577394 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:38077
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.129.keytab
--principal=oryx/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:32981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:27.024869 25038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:27.025300 25038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:27.025413 25038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:27.033754 25038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:27.033949 25038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:27.034140 25038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:34:27.050899 25038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:32981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.129.keytab
--principal=oryx/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:38077
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:27.052860 25038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:27.054620 25038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:27.076524 25046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:27.077201 25048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:27.077580 25044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:27.078696 25038 server_base.cc:1034] running on GCE node
I20260806 22:34:27.079756 25038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:27.081104 25038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:27.082557 25038 hybrid_clock.cc:648] HybridClock initialized: now 1786055667082470 us; error 82 us; skew 500 ppm
I20260806 22:34:27.083053 25038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:27 dist-test-slave-h3ft krb5kdc[24884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055667, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:27.087988 25038 init.cc:377] Logged in from keytab as oryx/127.15.198.129@KRBTEST.COM (short username oryx)
I20260806 22:34:27.090826 25038 webserver.cc:460] Webserver started at http://127.15.198.129:44815/ using document root <none> and password file <none>
I20260806 22:34:27.092052 25038 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:27.092228 25038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:27.092674 25038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:27.096817 25038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "cc4630b0558449a09775c2ad06c9daf8"
format_stamp: "Formatted at 2026-08-06 22:34:27 on dist-test-slave-h3ft"
server_key: "2fd9ea615a60ece918a1d0344d9b9783"
server_key_iv: "28c7dfcd913f36554459528a9f41d6c8"
server_key_version: "encryptionkey@0"
I20260806 22:34:27.098053 25038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "cc4630b0558449a09775c2ad06c9daf8"
format_stamp: "Formatted at 2026-08-06 22:34:27 on dist-test-slave-h3ft"
server_key: "2fd9ea615a60ece918a1d0344d9b9783"
server_key_iv: "28c7dfcd913f36554459528a9f41d6c8"
server_key_version: "encryptionkey@0"
I20260806 22:34:27.105581 25038 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.004s
I20260806 22:34:27.110327 25059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:27.112452 25038 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 22:34:27.112730 25038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "cc4630b0558449a09775c2ad06c9daf8"
format_stamp: "Formatted at 2026-08-06 22:34:27 on dist-test-slave-h3ft"
server_key: "2fd9ea615a60ece918a1d0344d9b9783"
server_key_iv: "28c7dfcd913f36554459528a9f41d6c8"
server_key_version: "encryptionkey@0"
I20260806 22:34:27.113073 25038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:27.141853 25038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:27.147720 25038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:27.148095 25038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:27.149461 25038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:27.151361 25038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:27.151484 25038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:27.151645 25038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:27.151729 25038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:27.195393 25038 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:45833
I20260806 22:34:27.195408 25214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:45833 every 8 connection(s)
I20260806 22:34:27.197219 25038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:34:27.199144 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 25038
I20260806 22:34:27.199446 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:34:27.200193 16154 external_mini_cluster.cc:1428] Setting key 05f3c04b704ac6c3328bfa1e67b1bda9
Aug 06 22:34:27 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055667, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM
I20260806 22:34:27.229806 25221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:27.201407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:45445 (local address 127.15.198.190:38077)
0806 22:34:27.202295 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:27.202303 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:27.206067 (+ 3764us) server_negotiation.cc:407] Connection header received
0806 22:34:27.207468 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:27.207479 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:27.207608 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:27.207765 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:27.211076 (+ 3311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:27.212553 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:27.215888 (+ 3335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:27.216731 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:27.216903 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:27.222591 (+ 5688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:27.222596 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:27.222600 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:27.222645 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:27.225776 (+ 3131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:27.226746 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:27.226751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:27.226753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:27.226922 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:27.227495 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:27.227500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:27.227502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:27.227874 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:27.228062 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:27.229021 (+ 959us) server_negotiation.cc:299] Negotiation successful
0806 22:34:27.229337 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":1252,"threads_started":1}
I20260806 22:34:27.231465 25223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:27.202400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.129:45445)
0806 22:34:27.203536 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:27.203609 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:27.207047 (+ 3438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:27.208123 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:27.208142 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:27.209433 (+ 1291us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:27.210700 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:27.210739 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:27.212885 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:27.212894 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:27.215582 (+ 2688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:27.215612 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:27.217566 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:27.217576 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:27.218130 (+ 554us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:27.219328 (+ 1198us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:27.219358 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:27.222237 (+ 2879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:27.225975 (+ 3738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:27.225987 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:27.226012 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:27.226579 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:27.227085 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:27.227090 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:27.227094 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:27.227350 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:27.228042 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:27.228052 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:27.228571 (+ 519us) client_negotiation.cc:769] Sending connection context
0806 22:34:27.228904 (+ 333us) client_negotiation.cc:240] Negotiation successful
0806 22:34:27.229649 (+ 745us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"thread_start_us":1006,"threads_started":1}
I20260806 22:34:27.233953 25216 heartbeater.cc:347] Connected to a master server at 127.15.198.190:38077
I20260806 22:34:27.234423 25216 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:27.235735 25216 heartbeater.cc:511] Master 127.15.198.190:38077 requested a full tablet report, sending...
I20260806 22:34:27.239311 24953 ts_manager.cc:194] Registered new tserver with Master: cc4630b0558449a09775c2ad06c9daf8 (127.15.198.129:45833)
I20260806 22:34:27.241266 24953 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:45445
WARNING: no policy specified for oryx/127.15.198.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.198.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.130.keytab.
Entry for principal oryx/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.130.keytab.
I20260806 22:34:27.299876 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:38077
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.130.keytab
--principal=oryx/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:32981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:27.771169 25226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:27.771628 25226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:27.771837 25226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:27.780666 25226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:27.780862 25226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:27.781080 25226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:34:27.794061 25226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:32981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.130.keytab
--principal=oryx/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:38077
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:27.796129 25226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:27.798161 25226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:27.815589 25234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:27.815749 25233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:27.815768 25238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:27.817457 25226 server_base.cc:1034] running on GCE node
I20260806 22:34:27.818392 25226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:27.819788 25226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:27.821044 25226 hybrid_clock.cc:648] HybridClock initialized: now 1786055667820959 us; error 76 us; skew 500 ppm
I20260806 22:34:27.821458 25226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:27 dist-test-slave-h3ft krb5kdc[24884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055667, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:27.825685 25226 init.cc:377] Logged in from keytab as oryx/127.15.198.130@KRBTEST.COM (short username oryx)
I20260806 22:34:27.827816 25226 webserver.cc:460] Webserver started at http://127.15.198.130:46191/ using document root <none> and password file <none>
I20260806 22:34:27.828824 25226 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:27.829005 25226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:27.829427 25226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:27.834153 25226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "7138c95721d2478f946f0a793fb57b8c"
format_stamp: "Formatted at 2026-08-06 22:34:27 on dist-test-slave-h3ft"
server_key: "29f990f22b55939f0636c0c4049654c0"
server_key_iv: "dcedce4f2edc9d1d1a81f90d788cffa6"
server_key_version: "encryptionkey@0"
I20260806 22:34:27.835426 25226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "7138c95721d2478f946f0a793fb57b8c"
format_stamp: "Formatted at 2026-08-06 22:34:27 on dist-test-slave-h3ft"
server_key: "29f990f22b55939f0636c0c4049654c0"
server_key_iv: "dcedce4f2edc9d1d1a81f90d788cffa6"
server_key_version: "encryptionkey@0"
I20260806 22:34:27.842924 25226 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260806 22:34:27.848390 25245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:27.850467 25226 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:34:27.850751 25226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "7138c95721d2478f946f0a793fb57b8c"
format_stamp: "Formatted at 2026-08-06 22:34:27 on dist-test-slave-h3ft"
server_key: "29f990f22b55939f0636c0c4049654c0"
server_key_iv: "dcedce4f2edc9d1d1a81f90d788cffa6"
server_key_version: "encryptionkey@0"
I20260806 22:34:27.851089 25226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:27.893455 25226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:27.899864 25226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:27.900280 25226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:27.901708 25226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:27.903770 25226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:27.903891 25226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:27.904076 25226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:27.904155 25226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:27.945333 25226 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:40791
I20260806 22:34:27.945370 25413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:40791 every 8 connection(s)
I20260806 22:34:27.947110 25226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:34:27.956542 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 25226
I20260806 22:34:27.956826 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:34:27.957530 16154 external_mini_cluster.cc:1428] Setting key 03d3bad8017fb9b52c1ceaee2ebc7eea
Aug 06 22:34:27 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055667, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM
I20260806 22:34:27.974128 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:27.950414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:56095 (local address 127.15.198.190:38077)
0806 22:34:27.951095 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:27.951100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:27.952264 (+ 1164us) server_negotiation.cc:407] Connection header received
0806 22:34:27.954510 (+ 2246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:27.954517 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:27.954633 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:27.954774 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:27.957420 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:27.958451 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:27.960390 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:27.960902 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:27.961056 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:27.966185 (+ 5129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:27.966192 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:27.966196 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:27.966246 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:27.969766 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:27.970923 (+ 1157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:27.970929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:27.970932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:27.971192 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:27.971763 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:27.971768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:27.971771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:27.972124 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:27.972340 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:27.973299 (+ 959us) server_negotiation.cc:299] Negotiation successful
0806 22:34:27.973640 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":298,"threads_started":1}
I20260806 22:34:27.975661 25422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:27.951210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.130:56095)
0806 22:34:27.952057 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:27.952106 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:27.954153 (+ 2047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:27.955124 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:27.955135 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:27.955891 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:27.957102 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:27.957136 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:27.958667 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:27.958672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:27.960212 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:27.960229 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:27.961204 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:27.961212 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:27.961519 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:27.962420 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:27.962459 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:27.965756 (+ 3297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:27.970018 (+ 4262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:27.970046 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:27.970068 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:27.970728 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:27.971338 (+ 610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:27.971346 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:27.971350 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:27.971609 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:27.972288 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:27.972298 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:27.972834 (+ 536us) client_negotiation.cc:769] Sending connection context
0806 22:34:27.973187 (+ 353us) client_negotiation.cc:240] Negotiation successful
0806 22:34:27.973988 (+ 801us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":313,"threads_started":1}
I20260806 22:34:27.977850 25416 heartbeater.cc:347] Connected to a master server at 127.15.198.190:38077
I20260806 22:34:27.978305 25416 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:27.979501 25416 heartbeater.cc:511] Master 127.15.198.190:38077 requested a full tablet report, sending...
I20260806 22:34:27.981707 24953 ts_manager.cc:194] Registered new tserver with Master: 7138c95721d2478f946f0a793fb57b8c (127.15.198.130:40791)
I20260806 22:34:27.982760 24953 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:56095
WARNING: no policy specified for oryx/127.15.198.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.198.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.131.keytab.
Entry for principal oryx/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.131.keytab.
I20260806 22:34:28.036465 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:38077
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.131.keytab
--principal=oryx/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:32981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:34:28.245721 25216 heartbeater.cc:503] Master 127.15.198.190:38077 was elected leader, sending a full tablet report...
W20260806 22:34:28.486181 25430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:28.486644 25430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:28.486833 25430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:28.495698 25430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:28.495878 25430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:28.496096 25430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:34:28.508280 25430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:32981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.131.keytab
--principal=oryx/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:38077
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:28.510227 25430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:28.512187 25430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:28.531407 25440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:28.531440 25438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:28.531399 25437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:28.532060 25430 server_base.cc:1034] running on GCE node
I20260806 22:34:28.533309 25430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:28.534708 25430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:28.536032 25430 hybrid_clock.cc:648] HybridClock initialized: now 1786055668535954 us; error 111 us; skew 500 ppm
I20260806 22:34:28.536550 25430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:28 dist-test-slave-h3ft krb5kdc[24884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055668, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:28.540910 25430 init.cc:377] Logged in from keytab as oryx/127.15.198.131@KRBTEST.COM (short username oryx)
I20260806 22:34:28.542963 25430 webserver.cc:460] Webserver started at http://127.15.198.131:42719/ using document root <none> and password file <none>
I20260806 22:34:28.543994 25430 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:28.544162 25430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:28.544557 25430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:28.548633 25430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "8f8024a031c4473aaff990b586743b03"
format_stamp: "Formatted at 2026-08-06 22:34:28 on dist-test-slave-h3ft"
server_key: "32c5e74f3fe9713ec5ee0e6a4eb32580"
server_key_iv: "5358fdc6c5677abe7b480115444dd14b"
server_key_version: "encryptionkey@0"
I20260806 22:34:28.549695 25430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "8f8024a031c4473aaff990b586743b03"
format_stamp: "Formatted at 2026-08-06 22:34:28 on dist-test-slave-h3ft"
server_key: "32c5e74f3fe9713ec5ee0e6a4eb32580"
server_key_iv: "5358fdc6c5677abe7b480115444dd14b"
server_key_version: "encryptionkey@0"
I20260806 22:34:28.556743 25430 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260806 22:34:28.561510 25450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:28.563714 25430 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:34:28.564008 25430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "8f8024a031c4473aaff990b586743b03"
format_stamp: "Formatted at 2026-08-06 22:34:28 on dist-test-slave-h3ft"
server_key: "32c5e74f3fe9713ec5ee0e6a4eb32580"
server_key_iv: "5358fdc6c5677abe7b480115444dd14b"
server_key_version: "encryptionkey@0"
I20260806 22:34:28.564322 25430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:28.579254 25430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:28.585398 25430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:28.585814 25430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:28.587172 25430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:28.589087 25430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:28.589210 25430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:28.589380 25430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:28.589454 25430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:28.632700 25430 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:44001
I20260806 22:34:28.632740 25612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:44001 every 8 connection(s)
I20260806 22:34:28.634476 25430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:34:28.636227 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 25430
I20260806 22:34:28.636520 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:34:28.637084 16154 external_mini_cluster.cc:1428] Setting key 18efcd6515c35b14efc4244064990faa
Aug 06 22:34:28 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055668, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM
I20260806 22:34:28.662400 25616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:28.638236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:40725 (local address 127.15.198.190:38077)
0806 22:34:28.639134 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:28.639142 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:28.640779 (+ 1637us) server_negotiation.cc:407] Connection header received
0806 22:34:28.643187 (+ 2408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:28.643193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:28.643292 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:28.643410 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:28.645711 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:28.646670 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:28.648770 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:28.649318 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:28.649461 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:28.654489 (+ 5028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:28.654500 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:28.654508 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:28.654577 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:28.658158 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:28.659313 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:28.659319 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:28.659321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:28.659493 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:28.660068 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:28.660073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:28.660075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:28.660450 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:28.660648 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:28.661580 (+ 932us) server_negotiation.cc:299] Negotiation successful
0806 22:34:28.661910 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":539,"threads_started":1}
I20260806 22:34:28.663885 25618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:28.639307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.131:40725)
0806 22:34:28.640562 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:28.640637 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:28.642812 (+ 2175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:28.643763 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:28.643776 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:28.644572 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:28.645468 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:28.645498 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:28.646882 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:28.646886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:28.648550 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:28.648576 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:28.649593 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:28.649598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:28.649857 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:28.650794 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:28.650825 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:28.654169 (+ 3344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:28.658391 (+ 4222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:28.658408 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:28.658436 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:28.659143 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:28.659633 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:28.659638 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:28.659642 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:28.659916 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:28.660599 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:28.660610 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:28.661128 (+ 518us) client_negotiation.cc:769] Sending connection context
0806 22:34:28.661467 (+ 339us) client_negotiation.cc:240] Negotiation successful
0806 22:34:28.662232 (+ 765us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"thread_start_us":478,"threads_started":1}
I20260806 22:34:28.666193 25613 heartbeater.cc:347] Connected to a master server at 127.15.198.190:38077
I20260806 22:34:28.666694 25613 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:28.667930 25613 heartbeater.cc:511] Master 127.15.198.190:38077 requested a full tablet report, sending...
I20260806 22:34:28.670233 24953 ts_manager.cc:194] Registered new tserver with Master: 8f8024a031c4473aaff990b586743b03 (127.15.198.131:44001)
I20260806 22:34:28.671495 24953 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:40725
I20260806 22:34:28.681701 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.198.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.198.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.198.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.189.keytab.
Entry for principal oryx/127.15.198.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.189.keytab.
I20260806 22:34:28.765496 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.189:44373
--webserver_interface=127.15.198.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.189.keytab
--principal=oryx/127.15.198.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:32981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:38077,127.15.198.189:44373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:34:28.986624 25416 heartbeater.cc:503] Master 127.15.198.190:38077 was elected leader, sending a full tablet report...
W20260806 22:34:29.238359 25621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:29.238834 25621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:29.238941 25621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:29.249115 25621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:34:29.249280 25621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:29.249359 25621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:29.249468 25621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:34:29.249553 25621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:34:29.262068 25621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:32981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:38077,127.15.198.189:44373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.189.keytab
--principal=oryx/127.15.198.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.189:44373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.198.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
W20260806 22:34:29.262650 25621 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.15.198.190:38077,127.15.198.189:44373'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 22:34:29.264317 25621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:29.266026 25621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:29.281193 25629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:29.281199 25635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:29.281311 25627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:29.282593 25621 server_base.cc:1034] running on GCE node
I20260806 22:34:29.283855 25621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:29.285892 25621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:29.287176 25621 hybrid_clock.cc:648] HybridClock initialized: now 1786055669287096 us; error 48 us; skew 500 ppm
I20260806 22:34:29.287657 25621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:29 dist-test-slave-h3ft krb5kdc[24884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055669, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:29.292255 25621 init.cc:377] Logged in from keytab as oryx/127.15.198.189@KRBTEST.COM (short username oryx)
I20260806 22:34:29.294495 25621 webserver.cc:460] Webserver started at http://127.15.198.189:32829/ using document root <none> and password file <none>
I20260806 22:34:29.295548 25621 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:29.295709 25621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:29.296128 25621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:29.300916 25621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/data/instance:
uuid: "b486dba97ea24ba7804d250a73a92c3b"
format_stamp: "Formatted at 2026-08-06 22:34:29 on dist-test-slave-h3ft"
server_key: "5fcc87acb8cd076d6b059ad2bdc45889"
server_key_iv: "961d1b525cfed69d84a0ce725e7d0181"
server_key_version: "encryptionkey@0"
I20260806 22:34:29.302240 25621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/wal/instance:
uuid: "b486dba97ea24ba7804d250a73a92c3b"
format_stamp: "Formatted at 2026-08-06 22:34:29 on dist-test-slave-h3ft"
server_key: "5fcc87acb8cd076d6b059ad2bdc45889"
server_key_iv: "961d1b525cfed69d84a0ce725e7d0181"
server_key_version: "encryptionkey@0"
I20260806 22:34:29.310333 25621 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.001s
I20260806 22:34:29.315721 25645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:29.318041 25621 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.002s
I20260806 22:34:29.318353 25621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/wal
uuid: "b486dba97ea24ba7804d250a73a92c3b"
format_stamp: "Formatted at 2026-08-06 22:34:29 on dist-test-slave-h3ft"
server_key: "5fcc87acb8cd076d6b059ad2bdc45889"
server_key_iv: "961d1b525cfed69d84a0ce725e7d0181"
server_key_version: "encryptionkey@0"
I20260806 22:34:29.318712 25621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:29.346925 25621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:29.352875 25621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:29.353233 25621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:29.378809 25621 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.189:44373
I20260806 22:34:29.378840 25724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.189:44373 every 8 connection(s)
I20260806 22:34:29.380867 25621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/data/info.pb
I20260806 22:34:29.385922 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 25621
I20260806 22:34:29.386500 25726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:29.400130 25726 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:34:29 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.198.189@KRBTEST.COM
Aug 06 22:34:29 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055669, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.189@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM
I20260806 22:34:29.420734 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:29.389051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45088 (local address 127.15.198.189:44373)
0806 22:34:29.390183 (+ 1132us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:29.390198 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:29.390260 (+ 62us) server_negotiation.cc:407] Connection header received
0806 22:34:29.393103 (+ 2843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:29.393142 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:29.393802 (+ 660us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:29.394382 (+ 580us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:29.397091 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.398863 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.401318 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.402385 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.402648 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:29.409235 (+ 6587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:29.409245 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:29.409260 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:29.409317 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:29.414143 (+ 4826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:29.415351 (+ 1208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:29.415363 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:29.415380 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:29.415620 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:29.416184 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:29.416189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:29.416191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:29.417067 (+ 876us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:29.417499 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:29.418192 (+ 693us) server_negotiation.cc:299] Negotiation successful
0806 22:34:29.418832 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":856,"thread_start_us":544,"threads_started":1}
I20260806 22:34:29.428061 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:29.404701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:53794)
0806 22:34:29.405889 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:29.405965 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:29.406193 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:29.408336 (+ 2143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:29.408346 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:29.409484 (+ 1138us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:29.409991 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:29.410008 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.411417 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.411424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:29.413060 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:29.413078 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.414325 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.414333 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:29.414626 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:29.415504 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:29.415530 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:29.419494 (+ 3964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:29.424457 (+ 4963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:29.424469 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:29.424478 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:29.425248 (+ 770us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:29.425805 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:29.425810 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:29.425813 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:29.426013 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:29.426781 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:29.426789 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:29.427107 (+ 318us) client_negotiation.cc:769] Sending connection context
0806 22:34:29.427329 (+ 222us) client_negotiation.cc:240] Negotiation successful
0806 22:34:29.427706 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"thread_start_us":948,"threads_started":1}
I20260806 22:34:29.428061 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:29.405501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53794 (local address 127.15.198.190:38077)
0806 22:34:29.407636 (+ 2135us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:29.407647 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:29.407677 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:34:29.407760 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:29.407767 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:29.407904 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:29.408055 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:29.410196 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.411223 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.413261 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.414018 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.414183 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:29.419795 (+ 5612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:29.419805 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:29.419809 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:29.419882 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:29.424265 (+ 4383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:29.425454 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:29.425462 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:29.425466 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:29.425662 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:29.426158 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:29.426164 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:29.426166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:29.426623 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:29.426770 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:29.427462 (+ 692us) server_negotiation.cc:299] Negotiation successful
0806 22:34:29.427727 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1967,"thread_start_us":1817,"threads_started":1}
I20260806 22:34:29.430646 25726 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:34:29 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055669, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.189@KRBTEST.COM for oryx/127.15.198.189@KRBTEST.COM
I20260806 22:34:29.447903 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:29.431826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:44373 (local address 127.0.0.1:45102)
0806 22:34:29.432222 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:29.432240 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:29.432362 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:29.432765 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:29.432770 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:29.433282 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:29.433466 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:29.433476 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.434731 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.434735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:29.435927 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:29.435943 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.436815 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.436820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:29.437009 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:29.437895 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:29.437919 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:29.440705 (+ 2786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:29.444846 (+ 4141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:29.444852 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:29.444855 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:29.445533 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:29.445945 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:29.445982 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:29.445984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:29.446132 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:29.446767 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:29.446771 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:29.446936 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 22:34:29.447251 (+ 315us) client_negotiation.cc:240] Negotiation successful
0806 22:34:29.447587 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272}
I20260806 22:34:29.448136 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:29.431826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45102 (local address 127.15.198.189:44373)
0806 22:34:29.432076 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:29.432081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:29.432345 (+ 264us) server_negotiation.cc:407] Connection header received
0806 22:34:29.432510 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:29.432515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:29.432597 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:29.432698 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:29.433639 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.434546 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.436153 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.436623 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.436750 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:29.440970 (+ 4220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:29.440978 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:29.440982 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:29.441025 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:29.444629 (+ 3604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:29.445678 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:29.445683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:29.445686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:29.445836 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:29.446253 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:29.446257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:29.446259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:29.446630 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:29.446762 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:29.447370 (+ 608us) server_negotiation.cc:299] Negotiation successful
0806 22:34:29.447773 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":143,"server-negotiator.queue_time_us":94}
I20260806 22:34:29.454955 25726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: Bootstrap starting.
I20260806 22:34:29.460391 25726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:29.461972 25726 log.cc:824] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:29.466275 25726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: No bootstrap required, opened a new log
I20260806 22:34:29.476241 25726 raft_consensus.cc:348] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } }
I20260806 22:34:29.476977 25726 raft_consensus.cc:374] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:29.477131 25726 raft_consensus.cc:729] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b486dba97ea24ba7804d250a73a92c3b, State: Initialized, Role: FOLLOWER
I20260806 22:34:29.478066 25726 consensus_queue.cc:260] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } }
I20260806 22:34:29.479362 25734 sys_catalog.cc:455] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } } }
I20260806 22:34:29.479804 25734 sys_catalog.cc:458] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:34:29.480966 25726 sys_catalog.cc:564] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: configured and running, proceeding with master startup.
W20260806 22:34:29.494670 25746 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:34:29.494984 25746 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:34:29.499184 25621 master_runner.cc:428] Detected that this master b486dba97ea24ba7804d250a73a92c3b is joining an existing cluster
I20260806 22:34:29.499363 25621 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.198.189:44373
I20260806 22:34:29.526249 25754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:29.506487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:44373 (local address 127.0.0.1:45118)
0806 22:34:29.507502 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:29.507522 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:29.507668 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:29.508225 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:29.508230 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:29.509060 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:29.509475 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:29.509488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.510903 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.510906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:29.512315 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:29.512330 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.513450 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.513458 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:29.513698 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:29.514700 (+ 1002us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:29.514728 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:29.516264 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:29.522092 (+ 5828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:29.522100 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:29.522105 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:29.522825 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:29.523647 (+ 822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:29.523652 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:29.523656 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:29.523910 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:29.524746 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:29.524752 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:29.525062 (+ 310us) client_negotiation.cc:769] Sending connection context
0806 22:34:29.525233 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 22:34:29.525637 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"mutex_wait_us":84,"thread_start_us":598,"threads_started":1}
I20260806 22:34:29.526786 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:29.505690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45118 (local address 127.15.198.189:44373)
0806 22:34:29.505920 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:29.505926 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:29.507636 (+ 1710us) server_negotiation.cc:407] Connection header received
0806 22:34:29.507807 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:29.507812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:29.507888 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:29.508011 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:29.509659 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.510678 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.512486 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.513211 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.513366 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:29.516514 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:29.516522 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:29.516528 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:29.516577 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:29.521831 (+ 5254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:29.523268 (+ 1437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:29.523275 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:29.523283 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:29.523499 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:29.524102 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:29.524110 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:29.524113 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:29.524612 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:29.524820 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:29.525391 (+ 571us) server_negotiation.cc:299] Negotiation successful
0806 22:34:29.526363 (+ 972us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":102,"server-negotiator.queue_time_us":65}
I20260806 22:34:29.527383 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:29.505742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53802 (local address 127.15.198.190:38077)
0806 22:34:29.506024 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:29.506031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:29.507104 (+ 1073us) server_negotiation.cc:407] Connection header received
0806 22:34:29.507540 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:29.507553 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:29.507675 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:29.507799 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:29.509159 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.510325 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.512750 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.513555 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.513698 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:29.516515 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:29.516522 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:29.516528 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:29.516584 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:29.521828 (+ 5244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:29.523903 (+ 2075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:29.523911 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:29.523916 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:29.524148 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:29.524617 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:29.524623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:29.524626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:29.525134 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:29.525312 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:29.526003 (+ 691us) server_negotiation.cc:299] Negotiation successful
0806 22:34:29.526956 (+ 953us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 22:34:29.529075 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:29.505761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:53802)
0806 22:34:29.506887 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:29.506916 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:29.507283 (+ 367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:29.507842 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:29.507847 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:29.508600 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:29.508986 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:29.508997 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.510529 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.510535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:29.512553 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:29.512584 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.513985 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.513991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:29.514130 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:29.515037 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:29.515069 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:29.516216 (+ 1147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:29.523056 (+ 6840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:29.523062 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:29.523066 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:29.523685 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:29.524327 (+ 642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:29.524332 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:29.524335 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:29.524483 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:29.525375 (+ 892us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:29.525384 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:29.525674 (+ 290us) client_negotiation.cc:769] Sending connection context
0806 22:34:29.527724 (+ 2050us) client_negotiation.cc:240] Negotiation successful
0806 22:34:29.528220 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":946,"mutex_wait_us":39,"thread_start_us":692,"threads_started":1}
I20260806 22:34:29.545594 25734 raft_consensus.cc:482] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:34:29.546124 25734 raft_consensus.cc:504] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } }
I20260806 22:34:29.548317 25734 leader_election.cc:288] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27fab5d8c68046e3973078cbdc8d5841 (127.15.198.190:38077)
I20260806 22:34:29.549626 24966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b486dba97ea24ba7804d250a73a92c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27fab5d8c68046e3973078cbdc8d5841" is_pre_election: true
Aug 06 22:34:29 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055666, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.190@KRBTEST.COM for oryx/127.15.198.189@KRBTEST.COM
I20260806 22:34:29.551460 25649 leader_election.cc:302] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: b486dba97ea24ba7804d250a73a92c3b; no voters: 27fab5d8c68046e3973078cbdc8d5841
I20260806 22:34:29.552599 25734 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 22:34:29.560326 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:29.537278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45134 (local address 127.15.198.189:44373)
0806 22:34:29.537533 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:29.537543 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:29.539960 (+ 2417us) server_negotiation.cc:407] Connection header received
0806 22:34:29.540499 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:29.540504 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:29.540661 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:29.540776 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:29.542848 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.543727 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.545802 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.546586 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.546720 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:29.552727 (+ 6007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:29.552738 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:29.552743 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:29.552801 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:29.556419 (+ 3618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:29.557612 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:29.557617 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:29.557620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:29.557807 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:29.558310 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:29.558315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:29.558317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:29.558711 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:29.558887 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:29.559536 (+ 649us) server_negotiation.cc:299] Negotiation successful
0806 22:34:29.559844 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 22:34:29.560384 25756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:29.537237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:44373 (local address 127.0.0.1:45134)
0806 22:34:29.539753 (+ 2516us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:29.539808 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:29.540191 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:29.541122 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:29.541134 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:29.542235 (+ 1101us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:29.542637 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:29.542654 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.543974 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.543979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:29.545565 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:29.545588 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.547114 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.547120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:29.547377 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:29.548365 (+ 988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:29.548403 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:29.552358 (+ 3955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:29.556624 (+ 4266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:29.556648 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:29.556659 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:29.557448 (+ 789us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:29.557939 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:29.557945 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:29.557956 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:29.558180 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:29.558862 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:29.558874 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:29.559184 (+ 310us) client_negotiation.cc:769] Sending connection context
0806 22:34:29.559411 (+ 227us) client_negotiation.cc:240] Negotiation successful
0806 22:34:29.559893 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2180,"thread_start_us":1999,"threads_started":1}
I20260806 22:34:29.563118 24953 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.198.189:44373: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b"
member_type: NON_VOTER
last_known_addr {
host: "127.15.198.189"
port: 44373
}
attrs {
promote: true
}
}
dest_uuid: "27fab5d8c68046e3973078cbdc8d5841"
cas_config_opid_index: -1
I20260806 22:34:29.564699 24953 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: NON_VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.15.198.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.198.188@KRBTEST.COM" created.
I20260806 22:34:29.575723 25764 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b486dba97ea24ba7804d250a73a92c3b (127.15.198.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: NON_VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: true } } }
I20260806 22:34:29.577814 25686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:29.582546 25764 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.198.189:44373
I20260806 22:34:29.582692 25686 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 1 FOLLOWER]: Refusing update from remote peer 27fab5d8c68046e3973078cbdc8d5841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 22:34:29.584103 25621 master.cc:537] Master@127.15.198.189:44373 shutting down...
I20260806 22:34:29.585889 25764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: NON_VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: true } } }
I20260806 22:34:29.586560 25764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:29.588203 25765 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: NON_VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:34:29.593106 24929 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 22:34:29.600045 24929 consensus_peers.cc:487] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 -> Peer b486dba97ea24ba7804d250a73a92c3b (127.15.198.189:44373): Couldn't send request to peer b486dba97ea24ba7804d250a73a92c3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 22:34:29.605747 25621 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:34:29.606613 25621 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:29.606993 25621 tablet_replica.cc:330] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: stopping tablet replica
I20260806 22:34:29.617244 25621 master.cc:552] Master@127.15.198.189:44373 shutdown complete.
I20260806 22:34:29.617486 25621 master_runner.cc:305] Clearing existing system tablet
I20260806 22:34:29.618198 25621 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:29.618348 25621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:29.621582 25621 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260806 22:34:29.623950 25772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:29.624415 25621 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:29.624747 25621 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 22:34:29.624845 25621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/wal
uuid: "b486dba97ea24ba7804d250a73a92c3b"
format_stamp: "Formatted at 2026-08-06 22:34:29 on dist-test-slave-h3ft"
server_key: "5fcc87acb8cd076d6b059ad2bdc45889"
server_key_iv: "961d1b525cfed69d84a0ce725e7d0181"
server_key_version: "encryptionkey@0"
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.198.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.188.keytab.
Entry for principal oryx/127.15.198.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.188.keytab.
I20260806 22:34:29.630335 25621 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:29.631971 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.188:37991
--webserver_interface=127.15.198.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.188.keytab
--principal=oryx/127.15.198.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:32981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:38077,127.15.198.189:44373,127.15.198.188:37991
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:34:29.640161 25621 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 22:34:29.640458 25621 log.cc:1196] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 22:34:29.641001 25621 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: Deleting consensus metadata
I20260806 22:34:29.641319 25621 master_runner.cc:315] Copying system tablet from 127.15.198.190:38077
I20260806 22:34:29.649333 25621 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.198.190:38077
I20260806 22:34:29.675227 25613 heartbeater.cc:503] Master 127.15.198.190:38077 was elected leader, sending a full tablet report...
I20260806 22:34:29.676363 25781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:29.650699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:53818)
0806 22:34:29.655654 (+ 4955us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:29.655691 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:29.655957 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:29.656537 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:29.656544 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:29.657399 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:29.657769 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:29.657786 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.659650 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.659656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:29.661346 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:29.661372 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.662636 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.662648 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:29.662860 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:29.663809 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:29.663845 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:29.665820 (+ 1975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:29.671455 (+ 5635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:29.671463 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:29.671471 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:29.672248 (+ 777us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:29.672912 (+ 664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:29.672920 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:29.672925 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:29.673180 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:29.673997 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:29.674011 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:29.674293 (+ 282us) client_negotiation.cc:769] Sending connection context
0806 22:34:29.674547 (+ 254us) client_negotiation.cc:240] Negotiation successful
0806 22:34:29.675540 (+ 993us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4740,"thread_start_us":4538,"threads_started":1}
I20260806 22:34:29.675666 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:29.650752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53818 (local address 127.15.198.190:38077)
0806 22:34:29.651070 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:29.651081 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:29.655897 (+ 4816us) server_negotiation.cc:407] Connection header received
0806 22:34:29.656202 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:29.656212 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:29.656338 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:29.656458 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:29.657999 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.659465 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.661558 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:29.662427 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:29.662599 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:29.666156 (+ 3557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:29.666166 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:29.666175 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:29.666245 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:29.671201 (+ 4956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:29.672468 (+ 1267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:29.672476 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:29.672480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:29.672733 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:29.673346 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:29.673352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:29.673356 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:29.673837 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:29.674024 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:29.674702 (+ 678us) server_negotiation.cc:299] Negotiation successful
0806 22:34:29.675068 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 22:34:29.678251 24982 tablet_copy_service.cc:140] P 27fab5d8c68046e3973078cbdc8d5841: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b486dba97ea24ba7804d250a73a92c3b ({username='oryx', principal='oryx/127.15.198.189@KRBTEST.COM'} at 127.0.0.1:53818)
I20260806 22:34:29.678740 24982 tablet_copy_service.cc:161] P 27fab5d8c68046e3973078cbdc8d5841: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b486dba97ea24ba7804d250a73a92c3b at {username='oryx', principal='oryx/127.15.198.189@KRBTEST.COM'} at 127.0.0.1:53818: session id = b486dba97ea24ba7804d250a73a92c3b-00000000000000000000000000000000
I20260806 22:34:29.684957 24982 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 22:34:29.689836 25621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 22:34:29.698781 25621 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: tablet copy: Starting download of 0 data blocks...
I20260806 22:34:29.699409 25621 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: tablet copy: Starting download of 1 WAL segments...
I20260806 22:34:29.704609 25621 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 22:34:29.710870 24982 tablet_copy_service.cc:342] P 27fab5d8c68046e3973078cbdc8d5841: Request end of tablet copy session b486dba97ea24ba7804d250a73a92c3b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.198.189@KRBTEST.COM'} at 127.0.0.1:53818
I20260806 22:34:29.711310 24982 tablet_copy_service.cc:433] P 27fab5d8c68046e3973078cbdc8d5841: ending tablet copy session b486dba97ea24ba7804d250a73a92c3b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b486dba97ea24ba7804d250a73a92c3b
I20260806 22:34:29.728967 25621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:29.732972 25782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:29.733608 25784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:29.743485 25621 server_base.cc:1034] running on GCE node
W20260806 22:34:29.744885 25786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:29.745628 25621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:29.746500 25621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:29.747737 25621 hybrid_clock.cc:648] HybridClock initialized: now 1786055669747663 us; error 45 us; skew 500 ppm
I20260806 22:34:29.748027 25621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:29 dist-test-slave-h3ft krb5kdc[24884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055669, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:29.750749 25621 init.cc:377] Logged in from keytab as oryx/127.15.198.189@KRBTEST.COM (short username oryx)
I20260806 22:34:29.756405 25621 webserver.cc:460] Webserver started at http://127.15.198.189:44681/ using document root <none> and password file <none>
I20260806 22:34:29.756889 25621 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:29.756983 25621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:29.761049 25621 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 22:34:29.763610 25796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:29.764156 25621 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 22:34:29.764318 25621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/wal
uuid: "b486dba97ea24ba7804d250a73a92c3b"
format_stamp: "Formatted at 2026-08-06 22:34:29 on dist-test-slave-h3ft"
server_key: "5fcc87acb8cd076d6b059ad2bdc45889"
server_key_iv: "961d1b525cfed69d84a0ce725e7d0181"
server_key_version: "encryptionkey@0"
I20260806 22:34:29.764562 25621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:29.793732 25621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:29.798897 25621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:29.847864 25621 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.189:44373
I20260806 22:34:29.847877 25866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.189:44373 every 8 connection(s)
I20260806 22:34:29.849720 25621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-1/data/info.pb
I20260806 22:34:29.856585 25867 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:34:29.861495 25867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: Bootstrap starting.
I20260806 22:34:29.887364 25867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:34:29.888206 25867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: Bootstrap complete.
I20260806 22:34:29.889804 25867 raft_consensus.cc:348] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: NON_VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: true } }
I20260806 22:34:29.890353 25867 raft_consensus.cc:729] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b486dba97ea24ba7804d250a73a92c3b, State: Initialized, Role: LEARNER
I20260806 22:34:29.890724 25867 consensus_queue.cc:260] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: NON_VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: true } }
I20260806 22:34:29.892055 25870 sys_catalog.cc:455] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: NON_VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: true } } }
I20260806 22:34:29.892513 25867 sys_catalog.cc:564] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:34:29.892765 25870 sys_catalog.cc:458] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: This master's current role is: LEARNER
I20260806 22:34:29.908313 25892 catalog_manager.cc:1259] Loaded cluster ID: c13aa0eae9f240ceb2a6470f0e8d05bb
I20260806 22:34:29.908653 25892 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: loading cluster ID for follower catalog manager: success
I20260806 22:34:29.913066 25892 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: acquiring CA information for follower catalog manager: success
I20260806 22:34:29.916641 25892 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:34:30.128918 25894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.118535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:44373 (local address 127.0.0.1:45138)
0806 22:34:30.119279 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:30.119312 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:30.119510 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:30.120432 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:30.120439 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:30.120496 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:30.120889 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.120944 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.123370 (+ 2426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.123375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.125185 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.125209 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.127868 (+ 2659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.127878 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.128196 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.128297 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 22:34:30.128413 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:34:30.128593 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":340,"threads_started":1}
I20260806 22:34:30.129722 25895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.118730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45138 (local address 127.15.198.189:44373)
0806 22:34:30.119691 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:30.119701 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:30.119735 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:34:30.119838 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:30.119844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:30.119981 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:30.120194 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:30.121141 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.123120 (+ 1979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.125475 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.127515 (+ 2040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.127849 (+ 334us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.128142 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:30.128502 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 22:34:30.128713 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":765,"thread_start_us":535,"threads_started":1}
I20260806 22:34:30.131503 25829 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260806 22:34:30.132403 25870 sys_catalog.cc:455] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27fab5d8c68046e3973078cbdc8d5841. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: NON_VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: true } } }
I20260806 22:34:30.132905 25870 sys_catalog.cc:458] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: This master's current role is: LEARNER
W20260806 22:34:30.214210 25777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:30.214851 25777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:30.215075 25777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:30.224511 25777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:34:30.224689 25777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:30.224767 25777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:30.224861 25777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:34:30.224951 25777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:34:30.238888 25777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:32981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:38077,127.15.198.189:44373,127.15.198.188:37991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.188.keytab
--principal=oryx/127.15.198.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.188:37991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.198.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:30.241052 25777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:30.243119 25777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:30.261790 25903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:30.261996 25902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:30.261914 25906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:30.263957 25777 server_base.cc:1034] running on GCE node
I20260806 22:34:30.265394 25777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:30.268227 25777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:30.269667 25777 hybrid_clock.cc:648] HybridClock initialized: now 1786055670269582 us; error 83 us; skew 500 ppm
I20260806 22:34:30.270308 25777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:30 dist-test-slave-h3ft krb5kdc[24884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055670, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:30.275755 25777 init.cc:377] Logged in from keytab as oryx/127.15.198.188@KRBTEST.COM (short username oryx)
I20260806 22:34:30.278365 25777 webserver.cc:460] Webserver started at http://127.15.198.188:40145/ using document root <none> and password file <none>
I20260806 22:34:30.279522 25777 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:30.279714 25777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:30.280187 25777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:30.285208 25777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/data/instance:
uuid: "5dd49501d6894cc49985c8633b52ff67"
format_stamp: "Formatted at 2026-08-06 22:34:30 on dist-test-slave-h3ft"
server_key: "64debaf031e1f944e8874ece48ad230b"
server_key_iv: "866457a6d4e11e23cce900533052a312"
server_key_version: "encryptionkey@0"
I20260806 22:34:30.286440 25777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/wal/instance:
uuid: "5dd49501d6894cc49985c8633b52ff67"
format_stamp: "Formatted at 2026-08-06 22:34:30 on dist-test-slave-h3ft"
server_key: "64debaf031e1f944e8874ece48ad230b"
server_key_iv: "866457a6d4e11e23cce900533052a312"
server_key_version: "encryptionkey@0"
I20260806 22:34:30.295428 25777 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.011s sys 0.000s
I20260806 22:34:30.301749 25915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:30.304077 25777 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:34:30.304370 25777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/wal
uuid: "5dd49501d6894cc49985c8633b52ff67"
format_stamp: "Formatted at 2026-08-06 22:34:30 on dist-test-slave-h3ft"
server_key: "64debaf031e1f944e8874ece48ad230b"
server_key_iv: "866457a6d4e11e23cce900533052a312"
server_key_version: "encryptionkey@0"
I20260806 22:34:30.304677 25777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:30.337648 25777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:30.343730 25777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:30.344136 25777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:30.372601 25777 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.188:37991
I20260806 22:34:30.372583 25985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.188:37991 every 8 connection(s)
I20260806 22:34:30.374797 25777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/data/info.pb
I20260806 22:34:30.380688 25986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:30.384110 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 25777
I20260806 22:34:30.392601 25986 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:34:30 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.198.188@KRBTEST.COM
Aug 06 22:34:30 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055670, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.188@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM
I20260806 22:34:30.421959 25991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.397375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:53832)
0806 22:34:30.398350 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:30.398415 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:30.398734 (+ 319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:30.399784 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:30.399794 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:30.401187 (+ 1393us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:30.401750 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.401766 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.403818 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.403824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.405265 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.405281 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.407116 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.407125 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.407347 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.408158 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:30.408195 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:30.411878 (+ 3683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:30.417591 (+ 5713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:30.417605 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:30.417615 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:30.418249 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:30.418868 (+ 619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:30.418874 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:30.418878 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:30.419102 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:30.419908 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:30.419918 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:30.420137 (+ 219us) client_negotiation.cc:769] Sending connection context
0806 22:34:30.420352 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 22:34:30.420747 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":407,"threads_started":1}
I20260806 22:34:30.421954 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.387862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40724 (local address 127.15.198.188:37991)
0806 22:34:30.389243 (+ 1381us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:30.389260 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:30.389325 (+ 65us) server_negotiation.cc:407] Connection header received
0806 22:34:30.392281 (+ 2956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:30.392387 (+ 106us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:30.393267 (+ 880us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:30.393976 (+ 709us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:30.396275 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.399947 (+ 3672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.401651 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.403264 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.403496 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.410536 (+ 7040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:30.410556 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:30.410586 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:30.410647 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:30.415919 (+ 5272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.417033 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.417043 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.417075 (+ 32us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.417334 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.417824 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.417829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.417832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.418702 (+ 870us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:30.419176 (+ 474us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:30.419797 (+ 621us) server_negotiation.cc:299] Negotiation successful
0806 22:34:30.420418 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1002,"thread_start_us":571,"threads_started":1}
I20260806 22:34:30.423038 25990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.397097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53832 (local address 127.15.198.190:38077)
0806 22:34:30.397910 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:30.397916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:30.398555 (+ 639us) server_negotiation.cc:407] Connection header received
0806 22:34:30.399247 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:30.399255 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:30.399369 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:30.399489 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:30.401950 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.403587 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.405458 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.406266 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.406440 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.412219 (+ 5779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:30.412229 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:30.412234 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:30.412328 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:30.417348 (+ 5020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.418517 (+ 1169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.418523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.418528 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.418727 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.419230 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.419235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.419239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.419673 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:30.419823 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:30.422036 (+ 2213us) server_negotiation.cc:299] Negotiation successful
0806 22:34:30.422396 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":430,"threads_started":1}
I20260806 22:34:30.424979 25986 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:34:30 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055670, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.188@KRBTEST.COM for oryx/127.15.198.189@KRBTEST.COM
I20260806 22:34:30.442512 25895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.426263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45142 (local address 127.15.198.189:44373)
0806 22:34:30.426498 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:30.426504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:30.426818 (+ 314us) server_negotiation.cc:407] Connection header received
0806 22:34:30.427078 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:30.427085 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:30.427175 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:30.427287 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:30.428196 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.428991 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.430361 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.430831 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.430988 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.435690 (+ 4702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:30.435698 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:30.435702 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:30.435752 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:30.439097 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.440184 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.440190 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.440193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.440388 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.440923 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.440928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.440931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.441338 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:30.441477 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:30.441948 (+ 471us) server_negotiation.cc:299] Negotiation successful
0806 22:34:30.442195 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 22:34:30.442683 25991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.426277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:44373 (local address 127.0.0.1:45142)
0806 22:34:30.426706 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:30.426726 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:30.426889 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:30.427319 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:30.427323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:30.427835 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:30.428017 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.428027 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.429131 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.429138 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.430192 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.430215 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.431033 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.431041 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.431279 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.432042 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:30.432066 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:30.435335 (+ 3269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:30.439350 (+ 4015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:30.439355 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:30.439359 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:30.439930 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:30.440571 (+ 641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:30.440575 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:30.440578 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:30.440776 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:30.441506 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:30.441510 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:30.441699 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 22:34:30.441849 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 22:34:30.442344 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242}
I20260806 22:34:30.444314 25986 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:34:30 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055670, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.188@KRBTEST.COM for oryx/127.15.198.188@KRBTEST.COM
I20260806 22:34:30.462646 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.445696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40732 (local address 127.15.198.188:37991)
0806 22:34:30.445932 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:30.445937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:30.446028 (+ 91us) server_negotiation.cc:407] Connection header received
0806 22:34:30.446228 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:30.446241 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:30.446309 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:30.446411 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:30.447533 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.448930 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.450320 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.451086 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.451224 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.455729 (+ 4505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:30.455735 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:30.455739 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:30.455780 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:30.459245 (+ 3465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.460053 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.460061 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.460065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.460310 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.460750 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.460757 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.460761 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.461120 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:30.461292 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:30.461764 (+ 472us) server_negotiation.cc:299] Negotiation successful
0806 22:34:30.462193 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":175,"server-negotiator.queue_time_us":73}
I20260806 22:34:30.462704 25991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.445656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37991 (local address 127.0.0.1:40732)
0806 22:34:30.445943 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:30.445963 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:30.446089 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:30.446471 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:30.446475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:30.447096 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:30.447367 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.447378 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.449098 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.449102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.450153 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.450167 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.451298 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.451303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.451551 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.452424 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:30.452444 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:30.455426 (+ 2982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:30.459445 (+ 4019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:30.459450 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:30.459455 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:30.459912 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:30.460436 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:30.460441 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:30.460447 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:30.460641 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:30.461240 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:30.461245 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:30.461491 (+ 246us) client_negotiation.cc:769] Sending connection context
0806 22:34:30.461655 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 22:34:30.462015 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 22:34:30.469679 25986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: Bootstrap starting.
I20260806 22:34:30.474421 25986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:30.476136 25986 log.cc:824] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:30.480355 25986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: No bootstrap required, opened a new log
I20260806 22:34:30.489759 25986 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } }
I20260806 22:34:30.490343 25986 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:30.490463 25986 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd49501d6894cc49985c8633b52ff67, State: Initialized, Role: FOLLOWER
I20260806 22:34:30.491276 25986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } }
I20260806 22:34:30.492805 25994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } } }
I20260806 22:34:30.493366 25994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:34:30.494280 25986 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 22:34:30.509865 26008 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:34:30.510140 26008 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:34:30.514884 25893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: NON_VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: true } } }
I20260806 22:34:30.515556 25893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:30.519063 25777 master_runner.cc:428] Detected that this master 5dd49501d6894cc49985c8633b52ff67 is joining an existing cluster
I20260806 22:34:30.519294 25777 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.198.188:37991
I20260806 22:34:30.526459 26011 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841: attempting to promote NON_VOTER b486dba97ea24ba7804d250a73a92c3b to VOTER
I20260806 22:34:30.528287 26011 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } }
I20260806 22:34:30.531447 25829 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 1 LEARNER]: Refusing update from remote peer 27fab5d8c68046e3973078cbdc8d5841: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 22:34:30.532095 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 25038
I20260806 22:34:30.532370 26012 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 22:34:30.540369 25893 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, b486dba97ea24ba7804d250a73a92c3b (127.15.198.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } }
I20260806 22:34:30.548913 25829 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, b486dba97ea24ba7804d250a73a92c3b (127.15.198.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } }
Aug 06 22:34:30 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.198.129@KRBTEST.COM
I20260806 22:34:30.558135 25870 sys_catalog.cc:455] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } }
I20260806 22:34:30.558564 25870 sys_catalog.cc:458] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:34:30.567966 26011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } }
I20260806 22:34:30.568495 26011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:30.569195 25893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } }
I20260806 22:34:30.569635 25893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:30.579475 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.534833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42794 (local address 127.15.198.129:45833)
0806 22:34:30.538565 (+ 3732us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:30.538578 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:30.538622 (+ 44us) server_negotiation.cc:407] Connection header received
0806 22:34:30.538733 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:30.538746 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:30.539191 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:30.542359 (+ 3168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:30.542768 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.544753 (+ 1985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.547786 (+ 3033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.549210 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.560870 (+ 11660us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.561352 (+ 482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:30.561372 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:30.561400 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:30.561452 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:30.572684 (+ 11232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.574349 (+ 1665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.574372 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.574377 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.574679 (+ 302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.575383 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.575389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.575393 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.576062 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:30.578094 (+ 2032us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:30.578337 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 22:34:30.578788 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3499,"thread_start_us":3148,"threads_started":1}
I20260806 22:34:30.581902 25209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:30.594098 25895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.535997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45146 (local address 127.15.198.189:44373)
0806 22:34:30.536243 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:30.536252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:30.558752 (+ 22500us) server_negotiation.cc:407] Connection header received
0806 22:34:30.558847 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:30.558852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:30.558969 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:30.559187 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:30.572328 (+ 13141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.573895 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.579245 (+ 5350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.579975 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.580087 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.585145 (+ 5058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:30.585153 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:30.585160 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:30.585226 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:30.589531 (+ 4305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.590925 (+ 1394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.590934 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.590939 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.591160 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.591740 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.591745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.591748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.592217 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:30.592368 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:30.593233 (+ 865us) server_negotiation.cc:299] Negotiation successful
0806 22:34:30.593606 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 22:34:30.595568 26024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.536658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:44373 (local address 127.0.0.1:45146)
0806 22:34:30.558129 (+ 21471us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:30.558160 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:30.558407 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:30.563149 (+ 4742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:30.563160 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:30.564949 (+ 1789us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:30.572025 (+ 7076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.572043 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.574176 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.574186 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.577102 (+ 2916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.577119 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.580815 (+ 3696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.580824 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.581692 (+ 868us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.583028 (+ 1336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:30.583058 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:30.584830 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:30.589807 (+ 4977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:30.589814 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:30.589818 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:30.590680 (+ 862us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:30.591304 (+ 624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:30.591310 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:30.591314 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:30.591594 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:30.592476 (+ 882us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:30.592481 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:30.592801 (+ 320us) client_negotiation.cc:769] Sending connection context
0806 22:34:30.594420 (+ 1619us) client_negotiation.cc:240] Negotiation successful
0806 22:34:30.594885 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20060,"mutex_wait_us":23,"spinlock_wait_cycles":51584,"thread_start_us":20411,"threads_started":1}
W20260806 22:34:30.596953 25923 tablet.cc:2369] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:34:30.606101 26026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.541763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37991 (local address 127.0.0.1:40748)
0806 22:34:30.571086 (+ 29323us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:30.571113 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:30.571515 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:30.572119 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:30.572126 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:30.574560 (+ 2434us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:30.575410 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.575425 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.578929 (+ 3504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.578935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.584203 (+ 5268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.584229 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.592398 (+ 8169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.592408 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.592758 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.593623 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:30.593647 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:30.595585 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:30.602203 (+ 6618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:30.602211 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:30.602215 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:30.602939 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:30.603484 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:30.603489 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:30.603493 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:30.603641 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:30.604594 (+ 953us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:30.604602 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:30.604818 (+ 216us) client_negotiation.cc:769] Sending connection context
0806 22:34:30.605000 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 22:34:30.605486 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29087,"mutex_wait_us":25,"thread_start_us":28242,"threads_started":1}
I20260806 22:34:30.607239 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.536497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40748 (local address 127.15.198.188:37991)
0806 22:34:30.536886 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:30.536892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:30.571232 (+ 34340us) server_negotiation.cc:407] Connection header received
0806 22:34:30.571678 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:30.571685 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:30.571779 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:30.571924 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:30.577327 (+ 5403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.578648 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.590762 (+ 12114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.591785 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.591933 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.596961 (+ 5028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:30.596970 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:30.596975 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:30.597021 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:30.601946 (+ 4925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.603182 (+ 1236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.603188 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.603192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.603357 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.603784 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.603789 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.603792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.604323 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:30.604511 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:30.606534 (+ 2023us) server_negotiation.cc:299] Negotiation successful
0806 22:34:30.606822 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":88}
I20260806 22:34:30.617551 25994 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:34:30.617975 25994 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } }
I20260806 22:34:30.620496 25990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.537383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53844 (local address 127.15.198.190:38077)
0806 22:34:30.537588 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:30.537593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:30.568336 (+ 30743us) server_negotiation.cc:407] Connection header received
0806 22:34:30.569310 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:30.569321 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:30.569453 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:30.569701 (+ 248us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:30.573978 (+ 4277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.575584 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.578120 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.578931 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.579120 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.584480 (+ 5360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:30.584489 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:30.584494 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:30.584551 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:30.589499 (+ 4948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.590852 (+ 1353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.590862 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.590868 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.591117 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.591739 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.591745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.591748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.592266 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:30.592393 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:30.603179 (+ 10786us) server_negotiation.cc:299] Negotiation successful
0806 22:34:30.604111 (+ 932us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 22:34:30.622802 26025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.540269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:53844)
0806 22:34:30.568167 (+ 27898us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:30.568201 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:30.568903 (+ 702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:30.570011 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:30.570020 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:30.573160 (+ 3140us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:30.573718 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.573736 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.575794 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.575800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.577881 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.577898 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.579555 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.579563 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.579795 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.580898 (+ 1103us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:30.580925 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:30.584130 (+ 3205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:30.589807 (+ 5677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:30.589814 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:30.589818 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:30.590680 (+ 862us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:30.591304 (+ 624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:30.591310 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:30.591314 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:30.591590 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:30.592481 (+ 891us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:30.592485 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:30.602407 (+ 9922us) client_negotiation.cc:769] Sending connection context
0806 22:34:30.621545 (+ 19138us) client_negotiation.cc:240] Negotiation successful
0806 22:34:30.622183 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25603,"mutex_wait_us":128,"spinlock_wait_cycles":4992,"thread_start_us":25242,"threads_started":1}
I20260806 22:34:30.648331 25994 leader_election.cc:288] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27fab5d8c68046e3973078cbdc8d5841 (127.15.198.190:38077), b486dba97ea24ba7804d250a73a92c3b (127.15.198.189:44373)
I20260806 22:34:30.649667 24966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5dd49501d6894cc49985c8633b52ff67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27fab5d8c68046e3973078cbdc8d5841" is_pre_election: true
I20260806 22:34:30.651432 25829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5dd49501d6894cc49985c8633b52ff67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b486dba97ea24ba7804d250a73a92c3b" is_pre_election: true
I20260806 22:34:30.653713 25916 leader_election.cc:302] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5dd49501d6894cc49985c8633b52ff67; no voters: 27fab5d8c68046e3973078cbdc8d5841, b486dba97ea24ba7804d250a73a92c3b
I20260806 22:34:30.654706 25994 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 06 22:34:30 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055666, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.190@KRBTEST.COM for oryx/127.15.198.188@KRBTEST.COM
I20260806 22:34:30.679502 26031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.654627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37991 (local address 127.0.0.1:40754)
0806 22:34:30.655509 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:30.655534 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:30.655709 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:30.656826 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:30.656831 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:30.657737 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:30.658094 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.658114 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.659921 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.659927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.661717 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.661737 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.663110 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.663119 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.663424 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.664480 (+ 1056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:30.664509 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:30.668335 (+ 3826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:30.674798 (+ 6463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:30.674807 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:30.674812 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:30.675744 (+ 932us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:30.676370 (+ 626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:30.676376 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:30.676380 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:30.676636 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:30.677476 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:30.677484 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:30.677706 (+ 222us) client_negotiation.cc:769] Sending connection context
0806 22:34:30.678528 (+ 822us) client_negotiation.cc:240] Negotiation successful
0806 22:34:30.678977 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":591,"threads_started":1}
I20260806 22:34:30.680580 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.655165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40754 (local address 127.15.198.188:37991)
0806 22:34:30.655560 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:30.655567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:30.655617 (+ 50us) server_negotiation.cc:407] Connection header received
0806 22:34:30.655911 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:30.655918 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:30.656062 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:30.656285 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:30.658347 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.659754 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.661926 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.662771 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.662954 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.668704 (+ 5750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:30.668713 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:30.668720 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:30.668791 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:30.674495 (+ 5704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.675999 (+ 1504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.676006 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.676010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.676210 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.676809 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.676815 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.676818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.677316 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:30.679241 (+ 1925us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:30.679435 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:34:30.680173 (+ 738us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249}
I20260806 22:34:30.682467 24953 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.198.188:37991: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5dd49501d6894cc49985c8633b52ff67"
member_type: NON_VOTER
last_known_addr {
host: "127.15.198.188"
port: 37991
}
attrs {
promote: true
}
}
dest_uuid: "27fab5d8c68046e3973078cbdc8d5841"
cas_config_opid_index: 6
I20260806 22:34:30.683746 24953 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: NON_VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: true } }
I20260806 22:34:30.687505 25829 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 1 FOLLOWER]: Refusing update from remote peer 27fab5d8c68046e3973078cbdc8d5841: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 22:34:30.688004 25953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:30.693305 25953 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 1 FOLLOWER]: Refusing update from remote peer 27fab5d8c68046e3973078cbdc8d5841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 22:34:30.694660 25893 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 22:34:30.695719 26011 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: NON_VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:30.699373 26028 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 5dd49501d6894cc49985c8633b52ff67 (127.15.198.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: NON_VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: true } } }
I20260806 22:34:30.705192 26028 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.198.188:37991
I20260806 22:34:30.704746 25829 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 5dd49501d6894cc49985c8633b52ff67 (127.15.198.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: NON_VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: true } } }
I20260806 22:34:30.705806 25893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: NON_VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: true } } }
I20260806 22:34:30.706080 25777 master.cc:537] Master@127.15.198.188:37991 shutting down...
I20260806 22:34:30.706106 25893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:30.706483 25893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: NON_VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: true } } }
I20260806 22:34:30.706776 25893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:30.727870 25870 sys_catalog.cc:455] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: NON_VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: true } } }
I20260806 22:34:30.728453 25870 sys_catalog.cc:458] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:34:30.732178 25953 raft_consensus.cc:858] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: NON_VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } }
W20260806 22:34:30.732468 25953 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: NON_VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } }
I20260806 22:34:30.746510 25953 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b486dba97ea24ba7804d250a73a92c3b (127.15.198.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: NON_VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: true } } }
I20260806 22:34:30.755580 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 25038
I20260806 22:34:30.756556 25994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27fab5d8c68046e3973078cbdc8d5841. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: NON_VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: true } } }
I20260806 22:34:30.756940 25994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 22:34:30.757192 25994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: NON_VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: true } } }
I20260806 22:34:30.757365 25994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 22:34:30.759904 26011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: NON_VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: true } } }
I20260806 22:34:30.760264 26011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:30.756644 26034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: NON_VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: true } } }
I20260806 22:34:30.767769 26034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: This master's current role is: NON_PARTICIPANT
W20260806 22:34:30.769932 24931 consensus_peers.cc:487] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 -> Peer 5dd49501d6894cc49985c8633b52ff67 (127.15.198.188:37991): Couldn't send request to peer 5dd49501d6894cc49985c8633b52ff67. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 22:34:30.772095 25777 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260806 22:34:30.772910 25777 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260806 22:34:30.773195 25777 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: stopping tablet replica
I20260806 22:34:30.794797 25777 master.cc:552] Master@127.15.198.188:37991 shutdown complete.
I20260806 22:34:30.795136 25777 master_runner.cc:305] Clearing existing system tablet
I20260806 22:34:30.795784 25777 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:30.795959 25777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:30.799957 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:45833
--local_ip_for_outbound_sockets=127.15.198.129
--tserver_master_addrs=127.15.198.190:38077
--webserver_port=44815
--webserver_interface=127.15.198.129
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.129.keytab
--principal=oryx/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:32981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.198.190:38077,127.15.198.189:44373
--tserver_master_addrs=127.15.198.190:38077,127.15.198.189:44373,127.15.198.188:37991 with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:34:30.801498 25777 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:34:30.803835 26046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:30.804419 25777 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:30.804747 25777 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 22:34:30.804877 25777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/wal
uuid: "5dd49501d6894cc49985c8633b52ff67"
format_stamp: "Formatted at 2026-08-06 22:34:30 on dist-test-slave-h3ft"
server_key: "64debaf031e1f944e8874ece48ad230b"
server_key_iv: "866457a6d4e11e23cce900533052a312"
server_key_version: "encryptionkey@0"
I20260806 22:34:30.809664 25777 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:30.817867 25777 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 22:34:30.818159 25777 log.cc:1196] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 22:34:30.818840 25777 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: Deleting consensus metadata
I20260806 22:34:30.819244 25777 master_runner.cc:315] Copying system tablet from 127.15.198.190:38077
I20260806 22:34:30.825191 25777 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: tablet copy: Beginning tablet copy session from remote peer at address 127.15.198.190:38077
I20260806 22:34:30.845579 26056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.826271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:53846)
0806 22:34:30.826918 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:30.826938 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:30.827200 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:30.827664 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:30.827669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:30.828248 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:30.828506 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.828517 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.829707 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.829718 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.831038 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:30.831060 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.832602 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.832612 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:30.832901 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.833934 (+ 1033us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:30.833968 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:30.835638 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:30.841510 (+ 5872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:30.841519 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:30.841526 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:30.842286 (+ 760us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:30.842932 (+ 646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:30.842939 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:30.842942 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:30.843185 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:30.844145 (+ 960us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:30.844151 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:30.844374 (+ 223us) client_negotiation.cc:769] Sending connection context
0806 22:34:30.844572 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 22:34:30.845045 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":350,"threads_started":1}
I20260806 22:34:30.847033 25990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:30.826370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53846 (local address 127.15.198.190:38077)
0806 22:34:30.826687 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:30.826695 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:30.827083 (+ 388us) server_negotiation.cc:407] Connection header received
0806 22:34:30.827368 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:30.827373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:30.827497 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:30.827616 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:30.828682 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.829532 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.831245 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:30.832037 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:30.832161 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:30.836020 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:30.836050 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:30.836056 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:30.836118 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:30.841128 (+ 5010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.842500 (+ 1372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.842506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.842510 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.842724 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:30.843338 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:30.843344 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:30.843348 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:30.843887 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:30.844074 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:30.846261 (+ 2187us) server_negotiation.cc:299] Negotiation successful
0806 22:34:30.846597 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260806 22:34:30.848584 24982 tablet_copy_service.cc:140] P 27fab5d8c68046e3973078cbdc8d5841: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5dd49501d6894cc49985c8633b52ff67 ({username='oryx', principal='oryx/127.15.198.188@KRBTEST.COM'} at 127.0.0.1:53846)
I20260806 22:34:30.848873 24982 tablet_copy_service.cc:161] P 27fab5d8c68046e3973078cbdc8d5841: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5dd49501d6894cc49985c8633b52ff67 at {username='oryx', principal='oryx/127.15.198.188@KRBTEST.COM'} at 127.0.0.1:53846: session id = 5dd49501d6894cc49985c8633b52ff67-00000000000000000000000000000000
I20260806 22:34:30.853708 24982 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 22:34:30.857821 25777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 22:34:30.875993 25777 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: tablet copy: Starting download of 0 data blocks...
I20260806 22:34:30.876744 25777 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: tablet copy: Starting download of 1 WAL segments...
I20260806 22:34:30.881479 25777 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 22:34:30.886796 24982 tablet_copy_service.cc:342] P 27fab5d8c68046e3973078cbdc8d5841: Request end of tablet copy session 5dd49501d6894cc49985c8633b52ff67-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.198.188@KRBTEST.COM'} at 127.0.0.1:53846
I20260806 22:34:30.887094 24982 tablet_copy_service.cc:433] P 27fab5d8c68046e3973078cbdc8d5841: ending tablet copy session 5dd49501d6894cc49985c8633b52ff67-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5dd49501d6894cc49985c8633b52ff67
I20260806 22:34:30.902000 25777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:30.905519 26057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:30.912004 26058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:30.916174 25777 server_base.cc:1034] running on GCE node
W20260806 22:34:30.916831 26060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:30.917379 25777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:30.918416 25777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:30.919639 25777 hybrid_clock.cc:648] HybridClock initialized: now 1786055670919587 us; error 49 us; skew 500 ppm
I20260806 22:34:30.919994 25777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:30 dist-test-slave-h3ft krb5kdc[24884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055670, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:30.922433 25777 init.cc:377] Logged in from keytab as oryx/127.15.198.188@KRBTEST.COM (short username oryx)
I20260806 22:34:30.928139 25777 webserver.cc:460] Webserver started at http://127.15.198.188:39161/ using document root <none> and password file <none>
I20260806 22:34:30.928596 25777 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:30.928730 25777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:30.932066 25777 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.002s
I20260806 22:34:30.934348 26071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:30.934880 25777 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 22:34:30.935086 25777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/wal
uuid: "5dd49501d6894cc49985c8633b52ff67"
format_stamp: "Formatted at 2026-08-06 22:34:30 on dist-test-slave-h3ft"
server_key: "64debaf031e1f944e8874ece48ad230b"
server_key_iv: "866457a6d4e11e23cce900533052a312"
server_key_version: "encryptionkey@0"
I20260806 22:34:30.935336 25777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:30.948608 25777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:30.953589 25777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:30.988552 25777 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.188:37991
I20260806 22:34:30.988567 26152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.188:37991 every 8 connection(s)
I20260806 22:34:30.990198 25777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/master-2/data/info.pb
I20260806 22:34:31.001271 26155 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:34:31.008098 26155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: Bootstrap starting.
I20260806 22:34:31.027196 26155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:34:31.027839 26155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: Bootstrap complete.
I20260806 22:34:31.028978 26155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: NON_VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: true } }
I20260806 22:34:31.029284 26155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5dd49501d6894cc49985c8633b52ff67, State: Initialized, Role: LEARNER
I20260806 22:34:31.029562 26155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: NON_VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: true } }
I20260806 22:34:31.031071 26155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:34:31.030674 26158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: NON_VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: true } } }
I20260806 22:34:31.031529 26158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:34:31.042629 26173 catalog_manager.cc:1259] Loaded cluster ID: c13aa0eae9f240ceb2a6470f0e8d05bb
I20260806 22:34:31.042796 26173 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: loading cluster ID for follower catalog manager: success
I20260806 22:34:31.045751 26173 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: acquiring CA information for follower catalog manager: success
I20260806 22:34:31.048224 26173 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:34:31.271509 25893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: NON_VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: true } } }
I20260806 22:34:31.271903 25893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:31.292774 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:31.272061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37991 (local address 127.0.0.1:40760)
0806 22:34:31.284261 (+ 12200us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:31.284300 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:31.284604 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:31.285194 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:31.285200 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:31.285258 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:31.285583 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:31.285600 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.287331 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.287338 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:31.289674 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:31.289698 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.291608 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.291690 (+ 82us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:31.291906 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:31.291946 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 22:34:31.292075 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 22:34:31.292256 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12003,"thread_start_us":11822,"threads_started":1}
I20260806 22:34:31.292809 26180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:31.272485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40760 (local address 127.15.198.188:37991)
0806 22:34:31.283844 (+ 11359us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:31.283857 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:31.284494 (+ 637us) server_negotiation.cc:407] Connection header received
0806 22:34:31.284803 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:31.284808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:31.284936 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:31.285095 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:31.285868 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.287126 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.289969 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.291300 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.291743 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:31.292007 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:31.292102 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:34:31.292275 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11053,"thread_start_us":11354,"threads_started":1}
I20260806 22:34:31.294277 26110 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260806 22:34:31.294826 26158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27fab5d8c68046e3973078cbdc8d5841. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: NON_VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: true } } }
I20260806 22:34:31.295256 26158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:34:31.295859 26175 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841: attempting to promote NON_VOTER 5dd49501d6894cc49985c8633b52ff67 to VOTER
I20260806 22:34:31.296653 26175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false } }
I20260806 22:34:31.299382 26110 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 1 LEARNER]: Refusing update from remote peer 27fab5d8c68046e3973078cbdc8d5841: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 22:34:31.299587 25829 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 1 FOLLOWER]: Refusing update from remote peer 27fab5d8c68046e3973078cbdc8d5841: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 22:34:31.300352 25893 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 22:34:31.300750 26174 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 22:34:31.304138 26174 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 5dd49501d6894cc49985c8633b52ff67 (127.15.198.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false } } }
I20260806 22:34:31.305065 25829 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 5dd49501d6894cc49985c8633b52ff67 (127.15.198.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false } } }
I20260806 22:34:31.306701 26110 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 5dd49501d6894cc49985c8633b52ff67 (127.15.198.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false } } }
I20260806 22:34:31.312227 26175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false } } }
I20260806 22:34:31.312263 26174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false } } }
I20260806 22:34:31.312912 26174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:31.313145 26175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:31.313030 26158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false } } }
I20260806 22:34:31.313174 26174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false } } }
I20260806 22:34:31.313416 26174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:31.313457 26158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:34:31.314967 26182 sys_catalog.cc:455] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "27fab5d8c68046e3973078cbdc8d5841" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false } } }
I20260806 22:34:31.315689 26182 sys_catalog.cc:458] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:34:31.410589 26047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:31.411178 26047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:31.411361 26047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:31.422386 26047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:31.422559 26047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:31.422797 26047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:34:31.436048 26047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:32981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.129.keytab
--principal=oryx/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:45833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=44815
--tserver_master_addrs=127.15.198.190:38077,127.15.198.189:44373,127.15.198.188:37991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:31.438076 26047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:31.440038 26047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:31.458773 26191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:31.458798 26193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:31.458832 26195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:31.464820 26047 server_base.cc:1034] running on GCE node
I20260806 22:34:31.465809 26047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:31.467199 26047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:31.468423 26047 hybrid_clock.cc:648] HybridClock initialized: now 1786055671468358 us; error 52 us; skew 500 ppm
I20260806 22:34:31.468817 26047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:31 dist-test-slave-h3ft krb5kdc[24884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055671, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:31.477090 26047 init.cc:377] Logged in from keytab as oryx/127.15.198.129@KRBTEST.COM (short username oryx)
I20260806 22:34:31.479171 26047 webserver.cc:460] Webserver started at http://127.15.198.129:44815/ using document root <none> and password file <none>
I20260806 22:34:31.480250 26047 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:31.480397 26047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:31.487784 26047 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 22:34:31.491626 26208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:31.493499 26047 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 22:34:31.493804 26047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "cc4630b0558449a09775c2ad06c9daf8"
format_stamp: "Formatted at 2026-08-06 22:34:27 on dist-test-slave-h3ft"
server_key: "2fd9ea615a60ece918a1d0344d9b9783"
server_key_iv: "28c7dfcd913f36554459528a9f41d6c8"
server_key_version: "encryptionkey@0"
I20260806 22:34:31.495276 26047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:31.527922 26047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:31.534299 26047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:31.534734 26047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:31.536226 26047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:31.538193 26047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:31.538321 26047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:31.538452 26047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:31.538552 26047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:31.588356 26047 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:45833
I20260806 22:34:31.588579 26373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:45833 every 8 connection(s)
I20260806 22:34:31.590382 26047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:34:31.597806 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 26047
I20260806 22:34:31.598296 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 25226
Aug 06 22:34:31 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055671, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.188@KRBTEST.COM
Aug 06 22:34:31 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.198.130@KRBTEST.COM
Aug 06 22:34:31 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055671, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM
Aug 06 22:34:31 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055671, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.189@KRBTEST.COM
I20260806 22:34:31.637490 26180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:31.598225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:55321 (local address 127.15.198.188:37991)
0806 22:34:31.598485 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:31.598495 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:31.605038 (+ 6543us) server_negotiation.cc:407] Connection header received
0806 22:34:31.606899 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:31.606906 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:31.607037 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:31.607127 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:31.610337 (+ 3210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.611699 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.617728 (+ 6029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.618576 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.618747 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:31.627560 (+ 8813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:31.627568 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:31.627573 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:31.627629 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:31.632663 (+ 5034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:31.634364 (+ 1701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:31.634371 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:31.634377 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:31.634637 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:31.635209 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:31.635215 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:31.635218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:31.635696 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:31.635849 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:31.636634 (+ 785us) server_negotiation.cc:299] Negotiation successful
0806 22:34:31.636937 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 22:34:31.639405 26387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:31.597565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37991 (local address 127.15.198.129:55321)
0806 22:34:31.604897 (+ 7332us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:31.604917 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:31.605164 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:31.607843 (+ 2679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:31.607865 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:31.609258 (+ 1393us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:31.610119 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:31.610143 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.611949 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.611955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:31.615465 (+ 3510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:31.615489 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.619429 (+ 3940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.619440 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:31.619829 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:31.620793 (+ 964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:31.620836 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:31.627157 (+ 6321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:31.632887 (+ 5730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:31.632892 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:31.632895 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:31.634181 (+ 1286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:31.634820 (+ 639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:31.634826 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:31.634831 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:31.635057 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:31.635952 (+ 895us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:31.635957 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:31.636152 (+ 195us) client_negotiation.cc:769] Sending connection context
0806 22:34:31.638180 (+ 2028us) client_negotiation.cc:240] Negotiation successful
0806 22:34:31.638515 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7176,"thread_start_us":6995,"threads_started":1}
I20260806 22:34:31.639964 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:31.595936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.129:41229)
0806 22:34:31.597556 (+ 1620us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:31.597618 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:31.602449 (+ 4831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:31.617335 (+ 14886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:31.617340 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:31.618125 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:31.618496 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:31.618514 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.620860 (+ 2346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.620866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:31.622239 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:31.622252 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.623509 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.623515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:31.623666 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:31.624363 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:31.624382 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:31.627253 (+ 2871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:31.632483 (+ 5230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:31.632500 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:31.632550 (+ 50us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:31.633472 (+ 922us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:31.634051 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:31.634057 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:31.634061 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:31.634343 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:31.635253 (+ 910us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:31.635264 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:31.635911 (+ 647us) client_negotiation.cc:769] Sending connection context
0806 22:34:31.636498 (+ 587us) client_negotiation.cc:240] Negotiation successful
0806 22:34:31.637273 (+ 775us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1099,"mutex_wait_us":20,"spinlock_wait_cycles":1920,"thread_start_us":9609,"threads_started":1}
I20260806 22:34:31.640441 26382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:31.594802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:41229 (local address 127.15.198.190:38077)
0806 22:34:31.616945 (+ 22143us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:31.616950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:31.616971 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:34:31.617069 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:31.617076 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:31.617169 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:31.617315 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:31.618739 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.620591 (+ 1852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.622397 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.623274 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.623424 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:31.627562 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:31.627570 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:31.627577 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:31.627639 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:31.632224 (+ 4585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:31.633610 (+ 1386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:31.633617 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:31.633620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:31.633848 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:31.634549 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:31.634558 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:31.634565 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:31.635089 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:31.639337 (+ 4248us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:31.639490 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 22:34:31.639852 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22004,"thread_start_us":21840,"threads_started":1}
I20260806 22:34:31.640959 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:31.594662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:53745 (local address 127.15.198.189:44373)
0806 22:34:31.595552 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:31.595558 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:31.616389 (+ 20831us) server_negotiation.cc:407] Connection header received
0806 22:34:31.616734 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:31.616741 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:31.616879 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:31.616985 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:31.618859 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.620591 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.625383 (+ 4792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.626157 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.626315 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:31.631615 (+ 5300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:31.631624 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:31.631630 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:31.631703 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:31.636898 (+ 5195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:31.638005 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:31.638013 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:31.638017 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:31.638329 (+ 312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:31.638840 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:31.638846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:31.638849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:31.639337 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:31.639494 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:31.640107 (+ 613us) server_negotiation.cc:299] Negotiation successful
0806 22:34:31.640425 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":401,"threads_started":1}
I20260806 22:34:31.642063 26391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:31.603516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:44373 (local address 127.15.198.129:53745)
0806 22:34:31.616246 (+ 12730us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:31.616272 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:31.616556 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:31.617312 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:31.617317 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:31.618158 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:31.618656 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:31.618669 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.620865 (+ 2196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.620870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:31.625192 (+ 4322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:31.625206 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.626699 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.626709 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:31.626930 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:31.627747 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:31.627775 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:31.631225 (+ 3450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:31.637325 (+ 6100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:31.637329 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:31.637333 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:31.637840 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:31.638503 (+ 663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:31.638508 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:31.638512 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:31.638696 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:31.639581 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:31.639585 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:31.639776 (+ 191us) client_negotiation.cc:769] Sending connection context
0806 22:34:31.641254 (+ 1478us) client_negotiation.cc:240] Negotiation successful
0806 22:34:31.641613 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12528,"mutex_wait_us":3,"spinlock_wait_cycles":41856,"thread_start_us":12238,"threads_started":1}
I20260806 22:34:31.642436 26376 heartbeater.cc:347] Connected to a master server at 127.15.198.190:38077
I20260806 22:34:31.642470 26375 heartbeater.cc:347] Connected to a master server at 127.15.198.188:37991
I20260806 22:34:31.642890 26376 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:31.642899 26375 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:31.643257 26377 heartbeater.cc:347] Connected to a master server at 127.15.198.189:44373
I20260806 22:34:31.643380 26377 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:31.644363 26376 heartbeater.cc:511] Master 127.15.198.190:38077 requested a full tablet report, sending...
I20260806 22:34:31.644388 26377 heartbeater.cc:511] Master 127.15.198.189:44373 requested a full tablet report, sending...
I20260806 22:34:31.644938 26375 heartbeater.cc:511] Master 127.15.198.188:37991 requested a full tablet report, sending...
I20260806 22:34:31.646991 24953 ts_manager.cc:194] Re-registered known tserver with Master: cc4630b0558449a09775c2ad06c9daf8 (127.15.198.129:45833)
I20260806 22:34:31.646942 26390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:31.602630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53556 (local address 127.15.198.130:40791)
0806 22:34:31.605866 (+ 3236us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:31.605874 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:31.605903 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:34:31.605976 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:31.605982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:31.606214 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:31.606408 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:31.610422 (+ 4014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.612205 (+ 1783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.614291 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:31.615870 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:31.616017 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:31.627067 (+ 11050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:31.627095 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:31.627122 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:31.627190 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:31.638148 (+ 10958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:31.639787 (+ 1639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:31.639802 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:31.639808 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:31.640075 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:31.642047 (+ 1972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:31.642053 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:31.642056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:31.644224 (+ 2168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:31.644648 (+ 424us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:31.646050 (+ 1402us) server_negotiation.cc:299] Negotiation successful
0806 22:34:31.646492 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3046,"thread_start_us":2835,"threads_started":1}
I20260806 22:34:31.647895 25817 ts_manager.cc:194] Registered new tserver with Master: cc4630b0558449a09775c2ad06c9daf8 (127.15.198.129:45833)
I20260806 22:34:31.648093 24953 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:41229
I20260806 22:34:31.649564 25408 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:31.650005 26095 ts_manager.cc:194] Registered new tserver with Master: cc4630b0558449a09775c2ad06c9daf8 (127.15.198.129:45833)
I20260806 22:34:31.747557 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 25226
I20260806 22:34:31.778139 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:40791
--local_ip_for_outbound_sockets=127.15.198.130
--tserver_master_addrs=127.15.198.190:38077
--webserver_port=46191
--webserver_interface=127.15.198.130
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.130.keytab
--principal=oryx/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:32981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.198.190:38077,127.15.198.189:44373
--tserver_master_addrs=127.15.198.190:38077,127.15.198.189:44373,127.15.198.188:37991 with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:32.223191 26393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:32.223608 26393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:32.223780 26393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:32.232677 26393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:32.232885 26393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:32.233074 26393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:34:32.244731 26393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:32981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.130.keytab
--principal=oryx/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:40791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=46191
--tserver_master_addrs=127.15.198.190:38077,127.15.198.189:44373,127.15.198.188:37991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:32.246709 26393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:32.248458 26393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:32.265736 26398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:32.265908 26401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:32.265763 26399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:32.266927 26393 server_base.cc:1034] running on GCE node
I20260806 22:34:32.267773 26393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:32.269083 26393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:32.270303 26393 hybrid_clock.cc:648] HybridClock initialized: now 1786055672270247 us; error 40 us; skew 500 ppm
I20260806 22:34:32.270718 26393 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:32 dist-test-slave-h3ft krb5kdc[24884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055672, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:32.274770 26393 init.cc:377] Logged in from keytab as oryx/127.15.198.130@KRBTEST.COM (short username oryx)
I20260806 22:34:32.276811 26393 webserver.cc:460] Webserver started at http://127.15.198.130:46191/ using document root <none> and password file <none>
I20260806 22:34:32.277719 26393 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:32.277909 26393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:32.285285 26393 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:34:32.289124 26414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:32.291406 26393 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:34:32.291651 26393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "7138c95721d2478f946f0a793fb57b8c"
format_stamp: "Formatted at 2026-08-06 22:34:27 on dist-test-slave-h3ft"
server_key: "29f990f22b55939f0636c0c4049654c0"
server_key_iv: "dcedce4f2edc9d1d1a81f90d788cffa6"
server_key_version: "encryptionkey@0"
I20260806 22:34:32.293084 26393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:32.319536 26393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:32.325358 26393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:32.325739 26393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:32.327147 26393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:32.329001 26393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:32.329105 26393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:32.329276 26393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:32.329378 26393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:32.371357 26393 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:40791
I20260806 22:34:32.371410 26598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:40791 every 8 connection(s)
I20260806 22:34:32.373243 26393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:34:32.378824 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 26393
I20260806 22:34:32.379225 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 25430
Aug 06 22:34:32 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055672, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.189@KRBTEST.COM
Aug 06 22:34:32 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055672, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.188@KRBTEST.COM
Aug 06 22:34:32 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055672, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM
I20260806 22:34:32.426435 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:32.377050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:59717 (local address 127.15.198.190:38077)
0806 22:34:32.389933 (+ 12883us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:32.389943 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:32.392189 (+ 2246us) server_negotiation.cc:407] Connection header received
0806 22:34:32.397892 (+ 5703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:32.397903 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:32.398034 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:32.398187 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:32.406576 (+ 8389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:32.407733 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:32.410530 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:32.411265 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:32.411417 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:32.418427 (+ 7010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:32.418434 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:32.418439 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:32.418496 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:32.423035 (+ 4539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:32.423879 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:32.423884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:32.423888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:32.424096 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:32.424680 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:32.424685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:32.424690 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:32.425113 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:32.425267 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:32.425790 (+ 523us) server_negotiation.cc:299] Negotiation successful
0806 22:34:32.426073 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12381,"thread_start_us":12538,"threads_started":1}
I20260806 22:34:32.427078 26609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:32.377966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.130:59717)
0806 22:34:32.391941 (+ 13975us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:32.392016 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:32.396111 (+ 4095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:32.398585 (+ 2474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:32.398598 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:32.406065 (+ 7467us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:32.406389 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:32.406402 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:32.408784 (+ 2382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:32.408790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:32.410358 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:32.410374 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:32.412723 (+ 2349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:32.412732 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:32.412959 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:32.414804 (+ 1845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:32.414836 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:32.418131 (+ 3295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:32.423164 (+ 5033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:32.423170 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:32.423175 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:32.423739 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:32.424249 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:32.424277 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:32.424282 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:32.424553 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:32.425279 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:32.425288 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:32.425506 (+ 218us) client_negotiation.cc:769] Sending connection context
0806 22:34:32.425667 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 22:34:32.426063 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13518,"thread_start_us":18742,"threads_started":1}
I20260806 22:34:32.427626 26606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:32.377106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:33221 (local address 127.15.198.189:44373)
0806 22:34:32.399416 (+ 22310us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:32.399426 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:32.399465 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:34:32.399550 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:32.399559 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:32.399677 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:32.399831 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:32.401914 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:32.403189 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:32.406613 (+ 3424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:32.407343 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:32.407491 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:32.415056 (+ 7565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:32.415067 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:32.415074 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:32.415137 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:32.419880 (+ 4743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:32.421244 (+ 1364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:32.421254 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:32.421260 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:32.421484 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:32.422141 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:32.422146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:32.422150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:32.422678 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:32.422836 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:32.426842 (+ 4006us) server_negotiation.cc:299] Negotiation successful
0806 22:34:32.427162 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22125,"thread_start_us":22922,"threads_started":1}
I20260806 22:34:32.427898 26608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:32.377966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:44373 (local address 127.15.198.130:33221)
0806 22:34:32.397332 (+ 19366us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:32.397354 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:32.397503 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:32.399857 (+ 2354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:32.399862 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:32.400527 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:32.401643 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:32.401678 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:32.403400 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:32.403405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:32.405856 (+ 2451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:32.405874 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:32.407868 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:32.407879 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:32.408246 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:32.410717 (+ 2471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:32.410759 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:32.414638 (+ 3879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:32.420152 (+ 5514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:32.420165 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:32.420192 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:32.421061 (+ 869us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:32.421652 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:32.421659 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:32.421663 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:32.421991 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:32.422965 (+ 974us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:32.422976 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:32.423519 (+ 543us) client_negotiation.cc:769] Sending connection context
0806 22:34:32.423997 (+ 478us) client_negotiation.cc:240] Negotiation successful
0806 22:34:32.424778 (+ 781us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19145,"mutex_wait_us":480,"thread_start_us":12778,"threads_started":1}
I20260806 22:34:32.428046 26612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:32.383904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37991 (local address 127.15.198.130:55569)
0806 22:34:32.398336 (+ 14432us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:32.398362 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:32.398548 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:32.401207 (+ 2659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:32.401215 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:32.402152 (+ 937us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:32.402591 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:32.402610 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:32.405539 (+ 2929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:32.405548 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:32.407892 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:32.407914 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:32.409741 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:32.409751 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:32.410047 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:32.411243 (+ 1196us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:32.411272 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:32.415622 (+ 4350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:32.421596 (+ 5974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:32.421602 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:32.421605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:32.423055 (+ 1450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:32.423628 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:32.423632 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:32.423638 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:32.424315 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:32.426920 (+ 2605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:32.426924 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:32.427128 (+ 204us) client_negotiation.cc:769] Sending connection context
0806 22:34:32.427274 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 22:34:32.427593 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14254,"thread_start_us":14739,"threads_started":1}
Aug 06 22:34:32 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.198.131@KRBTEST.COM
I20260806 22:34:32.429260 26613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:32.384576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:55569 (local address 127.15.198.188:37991)
0806 22:34:32.400744 (+ 16168us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:32.400754 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:32.400782 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:34:32.400883 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:32.400891 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:32.400987 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:32.402803 (+ 1816us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:32.403161 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:32.404590 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:32.408192 (+ 3602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:32.409095 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:32.409305 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:32.415980 (+ 6675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:32.415990 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:32.415998 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:32.416055 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:32.421398 (+ 5343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:32.423226 (+ 1828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:32.423234 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:32.423239 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:32.423467 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:32.424453 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:32.424458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:32.424461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:32.425613 (+ 1152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:32.426840 (+ 1227us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:32.428471 (+ 1631us) server_negotiation.cc:299] Negotiation successful
0806 22:34:32.428810 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15988,"thread_start_us":15829,"threads_started":1}
I20260806 22:34:32.430056 26602 heartbeater.cc:347] Connected to a master server at 127.15.198.190:38077
I20260806 22:34:32.430058 26603 heartbeater.cc:347] Connected to a master server at 127.15.198.189:44373
I20260806 22:34:32.430797 26601 heartbeater.cc:347] Connected to a master server at 127.15.198.188:37991
I20260806 22:34:32.430792 26603 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:32.430792 26602 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:32.431167 26601 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:32.432942 26601 heartbeater.cc:511] Master 127.15.198.188:37991 requested a full tablet report, sending...
I20260806 22:34:32.433019 26603 heartbeater.cc:511] Master 127.15.198.189:44373 requested a full tablet report, sending...
I20260806 22:34:32.433104 26602 heartbeater.cc:511] Master 127.15.198.190:38077 requested a full tablet report, sending...
I20260806 22:34:32.435392 24953 ts_manager.cc:194] Re-registered known tserver with Master: 7138c95721d2478f946f0a793fb57b8c (127.15.198.130:40791)
I20260806 22:34:32.435813 26095 ts_manager.cc:194] Registered new tserver with Master: 7138c95721d2478f946f0a793fb57b8c (127.15.198.130:40791)
I20260806 22:34:32.435859 25817 ts_manager.cc:194] Registered new tserver with Master: 7138c95721d2478f946f0a793fb57b8c (127.15.198.130:40791)
I20260806 22:34:32.436928 24953 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:59717
I20260806 22:34:32.440719 26611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:32.381305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40062 (local address 127.15.198.131:44001)
0806 22:34:32.405362 (+ 24057us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:32.405378 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:32.411485 (+ 6107us) server_negotiation.cc:407] Connection header received
0806 22:34:32.411607 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:32.411620 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:32.412018 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:32.412215 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:32.416298 (+ 4083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:32.418391 (+ 2093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:32.420526 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:32.421750 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:32.421902 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:32.430988 (+ 9086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:32.431072 (+ 84us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:32.431096 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:32.431168 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:32.437120 (+ 5952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:32.438076 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:32.438091 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:32.438094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:32.438269 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:32.438723 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:32.438729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:32.438732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:32.439245 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:32.439601 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:32.439938 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 22:34:32.440282 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23817,"thread_start_us":23590,"threads_started":1}
I20260806 22:34:32.442238 25608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:32.538551 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 25430
I20260806 22:34:32.569507 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:44001
--local_ip_for_outbound_sockets=127.15.198.131
--tserver_master_addrs=127.15.198.190:38077
--webserver_port=42719
--webserver_interface=127.15.198.131
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.131.keytab
--principal=oryx/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:32981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.198.190:38077,127.15.198.189:44373
--tserver_master_addrs=127.15.198.190:38077,127.15.198.189:44373,127.15.198.188:37991 with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:34:32.653456 26376 heartbeater.cc:503] Master 127.15.198.190:38077 was elected leader, sending a full tablet report...
W20260806 22:34:33.016779 26616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:33.017238 26616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:33.017427 26616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:33.026042 26616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:33.026233 26616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:33.026422 26616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:34:33.038308 26616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:32981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.131.keytab
--principal=oryx/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:44001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=42719
--tserver_master_addrs=127.15.198.190:38077,127.15.198.189:44373,127.15.198.188:37991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:33.040221 26616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:33.041953 26616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:33.058136 26621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:33.058192 26626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:33.058341 26623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:33.059744 26616 server_base.cc:1034] running on GCE node
I20260806 22:34:33.060653 26616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:33.062033 26616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:33.063321 26616 hybrid_clock.cc:648] HybridClock initialized: now 1786055673063175 us; error 117 us; skew 500 ppm
I20260806 22:34:33.063799 26616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:33 dist-test-slave-h3ft krb5kdc[24884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055673, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:33.068171 26616 init.cc:377] Logged in from keytab as oryx/127.15.198.131@KRBTEST.COM (short username oryx)
I20260806 22:34:33.070139 26616 webserver.cc:460] Webserver started at http://127.15.198.131:42719/ using document root <none> and password file <none>
I20260806 22:34:33.071161 26616 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:33.071344 26616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:33.078347 26616 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:34:33.082072 26637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:33.084010 26616 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:34:33.084244 26616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "8f8024a031c4473aaff990b586743b03"
format_stamp: "Formatted at 2026-08-06 22:34:28 on dist-test-slave-h3ft"
server_key: "32c5e74f3fe9713ec5ee0e6a4eb32580"
server_key_iv: "5358fdc6c5677abe7b480115444dd14b"
server_key_version: "encryptionkey@0"
I20260806 22:34:33.085630 26616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:33.106876 26616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:33.112841 26616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:33.113240 26616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:33.114630 26616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:33.116618 26616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:33.116755 26616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:33.116909 26616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:33.117002 26616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:33.159344 26616 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:44001
I20260806 22:34:33.159408 26803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:44001 every 8 connection(s)
I20260806 22:34:33.161515 26616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:34:33.167742 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 26616
Aug 06 22:34:33 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055673, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.188@KRBTEST.COM
Aug 06 22:34:33 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055673, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM
Aug 06 22:34:33 dist-test-slave-h3ft krb5kdc[24884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055673, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.189@KRBTEST.COM
I20260806 22:34:33.221056 26809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.165239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:59773 (local address 127.15.198.190:38077)
0806 22:34:33.185099 (+ 19860us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:33.185104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:33.185123 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:34:33.185217 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:33.185222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:33.185314 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:33.185443 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:33.189122 (+ 3679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.190201 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.194471 (+ 4270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.195535 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.195682 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.208750 (+ 13068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:33.208758 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:33.208765 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:33.208898 (+ 133us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:33.213185 (+ 4287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.217100 (+ 3915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.217110 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.217116 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.217325 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.218260 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.218265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.218269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.218751 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:33.218975 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:33.220242 (+ 1267us) server_negotiation.cc:299] Negotiation successful
0806 22:34:33.220608 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19731,"thread_start_us":19601,"threads_started":1}
I20260806 22:34:33.224412 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.165404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:37563 (local address 127.15.198.188:37991)
0806 22:34:33.179605 (+ 14201us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:33.179617 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:33.179657 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:34:33.184818 (+ 5161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:33.184830 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:33.185028 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:33.185220 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:33.189142 (+ 3922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.190473 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.195944 (+ 5471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.196961 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.197112 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.206615 (+ 9503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:33.206624 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:33.206629 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:33.206685 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:33.216855 (+ 10170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.217777 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.217784 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.217788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.217991 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.219089 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.219095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.219098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.219496 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:33.219662 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:33.223648 (+ 3986us) server_negotiation.cc:299] Negotiation successful
0806 22:34:33.223966 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14007,"thread_start_us":14255,"threads_started":1}
I20260806 22:34:33.224527 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.167104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37991 (local address 127.15.198.131:37563)
0806 22:34:33.175448 (+ 8344us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:33.175470 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:33.184127 (+ 8657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:33.185670 (+ 1543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:33.185687 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:33.187656 (+ 1969us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:33.188841 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:33.188850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.191183 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.191189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:33.195683 (+ 4494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:33.195707 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.198199 (+ 2492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.198208 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:33.198482 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.200131 (+ 1649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:33.200175 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:33.206285 (+ 6110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:33.217029 (+ 10744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:33.217036 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:33.217041 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:33.217633 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:33.218271 (+ 638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:33.218275 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:33.218278 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:33.218477 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:33.223104 (+ 4627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:33.223111 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:33.223332 (+ 221us) client_negotiation.cc:769] Sending connection context
0806 22:34:33.223499 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 22:34:33.223851 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8166,"mutex_wait_us":2536,"thread_start_us":8814,"threads_started":1}
I20260806 22:34:33.222946 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.166174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.131:59773)
0806 22:34:33.175323 (+ 9149us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:33.175438 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:33.184134 (+ 8696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:33.185670 (+ 1536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:33.185690 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:33.187128 (+ 1438us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:33.188785 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:33.188826 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.190733 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.190743 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:33.194207 (+ 3464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:33.194232 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.196226 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.196236 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:33.196753 (+ 517us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.198411 (+ 1658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:33.198474 (+ 63us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:33.208154 (+ 9680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:33.213403 (+ 5249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:33.213417 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:33.213444 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:33.214560 (+ 1116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:33.217781 (+ 3221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:33.217786 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:33.217789 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:33.218133 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:33.218930 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:33.218945 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:33.219662 (+ 717us) client_negotiation.cc:769] Sending connection context
0806 22:34:33.220110 (+ 448us) client_negotiation.cc:240] Negotiation successful
0806 22:34:33.220834 (+ 724us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8530,"mutex_wait_us":189,"spinlock_wait_cycles":31616,"thread_start_us":8152,"threads_started":1}
I20260806 22:34:33.227280 26804 heartbeater.cc:347] Connected to a master server at 127.15.198.188:37991
I20260806 22:34:33.227751 26804 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:33.228257 26805 heartbeater.cc:347] Connected to a master server at 127.15.198.190:38077
I20260806 22:34:33.228499 26805 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:33.229110 26804 heartbeater.cc:511] Master 127.15.198.188:37991 requested a full tablet report, sending...
I20260806 22:34:33.229297 26805 heartbeater.cc:511] Master 127.15.198.190:38077 requested a full tablet report, sending...
I20260806 22:34:33.232079 24953 ts_manager.cc:194] Re-registered known tserver with Master: 8f8024a031c4473aaff990b586743b03 (127.15.198.131:44001)
I20260806 22:34:33.232115 26095 ts_manager.cc:194] Registered new tserver with Master: 8f8024a031c4473aaff990b586743b03 (127.15.198.131:44001)
I20260806 22:34:33.233529 26821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.166175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:44373 (local address 127.15.198.131:50275)
0806 22:34:33.179573 (+ 13398us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:33.179599 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:33.184127 (+ 4528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:33.186201 (+ 2074us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:33.186205 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:33.187240 (+ 1035us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:33.188784 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:33.188826 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.192365 (+ 3539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.192373 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:33.203480 (+ 11107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:33.203505 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.206644 (+ 3139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.206651 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:33.206877 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.210149 (+ 3272us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:33.210177 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:33.215897 (+ 5720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:33.225582 (+ 9685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:33.225611 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:33.225615 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:33.226197 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:33.230792 (+ 4595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:33.230800 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:33.230804 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:33.231137 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:33.232107 (+ 970us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:33.232116 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:33.232393 (+ 277us) client_negotiation.cc:769] Sending connection context
0806 22:34:33.232574 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 22:34:33.233038 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13214,"mutex_wait_us":1315,"thread_start_us":11875,"threads_started":1}
I20260806 22:34:33.233459 24953 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:59773
I20260806 22:34:33.235100 26817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.166642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:50275 (local address 127.15.198.189:44373)
0806 22:34:33.183997 (+ 17355us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:33.184009 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:33.184051 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:34:33.185699 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:33.185705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:33.185825 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:33.185971 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:33.190521 (+ 4550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.192020 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.203745 (+ 11725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.204743 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.204866 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.216260 (+ 11394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:33.216273 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:33.216278 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:33.216374 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:33.222862 (+ 6488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.228807 (+ 5945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.228816 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.228820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.229140 (+ 320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.231289 (+ 2149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.231295 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.231299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.231922 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:33.232127 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:33.234143 (+ 2016us) server_negotiation.cc:299] Negotiation successful
0806 22:34:33.234546 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17130,"thread_start_us":17003,"threads_started":1}
I20260806 22:34:33.236128 26807 heartbeater.cc:347] Connected to a master server at 127.15.198.189:44373
I20260806 22:34:33.236301 26807 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:33.238343 26807 heartbeater.cc:511] Master 127.15.198.189:44373 requested a full tablet report, sending...
I20260806 22:34:33.241226 25817 ts_manager.cc:194] Registered new tserver with Master: 8f8024a031c4473aaff990b586743b03 (127.15.198.131:44001)
I20260806 22:34:33.252713 26839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.205329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40762 (local address 127.15.198.188:37991)
0806 22:34:33.224102 (+ 18773us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:33.224111 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:33.224135 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:34:33.224205 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:33.224211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:33.224347 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:33.224525 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:33.230876 (+ 6351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.234125 (+ 3249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.236638 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.237485 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.237629 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.242726 (+ 5097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:33.242735 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:33.242741 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:33.242802 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:33.247192 (+ 4390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.248188 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.248194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.248197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.248404 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.249047 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.249054 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.249057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.249721 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:33.250078 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:33.251684 (+ 1606us) server_negotiation.cc:299] Negotiation successful
0806 22:34:33.252079 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18504,"thread_start_us":19429,"threads_started":1}
I20260806 22:34:33.253997 26832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.191099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53858 (local address 127.15.198.190:38077)
0806 22:34:33.199636 (+ 8537us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:33.199646 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:33.206874 (+ 7228us) server_negotiation.cc:407] Connection header received
0806 22:34:33.207030 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:33.207041 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:33.207180 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:33.207340 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:33.226095 (+ 18755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.227749 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.237325 (+ 9576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.238240 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.238406 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.243394 (+ 4988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:33.243402 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:33.243408 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:33.243466 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:33.248290 (+ 4824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.249439 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.249445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.249449 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.249744 (+ 295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.250299 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.250304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.250307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.250959 (+ 652us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:33.251375 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:33.252684 (+ 1309us) server_negotiation.cc:299] Negotiation successful
0806 22:34:33.253504 (+ 820us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8310,"thread_start_us":8177,"threads_started":1}
I20260806 22:34:33.256778 26834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.191804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45162 (local address 127.15.198.189:44373)
0806 22:34:33.229569 (+ 37765us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:33.229577 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:33.229600 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:34:33.229660 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:33.229665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:33.229777 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:33.230520 (+ 743us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:33.237664 (+ 7144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.239119 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.241819 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.242588 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.242723 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.246807 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:33.246820 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:33.246829 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:33.246908 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:33.251765 (+ 4857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.253511 (+ 1746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.253517 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.253520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.253722 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.254349 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.254356 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.254359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.255053 (+ 694us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:33.255411 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:33.256117 (+ 706us) server_negotiation.cc:299] Negotiation successful
0806 22:34:33.256420 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37569,"thread_start_us":33365,"threads_started":1}
I20260806 22:34:33.260320 24953 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53858:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:34:33.271445 24953 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:34:33.313913 26856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.298097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:40791 (local address 127.0.0.1:53572)
0806 22:34:33.299707 (+ 1610us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:33.299734 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:33.299905 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:33.301319 (+ 1414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:33.301328 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:33.301376 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:33.302241 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:33.302264 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.305313 (+ 3049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.305321 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:33.308985 (+ 3664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:33.309012 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.312317 (+ 3305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.312324 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:33.312588 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.312641 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 22:34:33.313212 (+ 571us) client_negotiation.cc:240] Negotiation successful
0806 22:34:33.313429 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1424,"thread_start_us":12708,"threads_started":1}
I20260806 22:34:33.314795 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.298979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53572 (local address 127.15.198.130:40791)
0806 22:34:33.300190 (+ 1211us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:33.300206 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:33.300249 (+ 43us) server_negotiation.cc:407] Connection header received
0806 22:34:33.300416 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:33.300430 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:33.300972 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:33.301379 (+ 407us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:33.302594 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.305037 (+ 2443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.309317 (+ 4280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.311659 (+ 2342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.311899 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.312131 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:33.314249 (+ 2118us) server_negotiation.cc:299] Negotiation successful
0806 22:34:33.314520 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1020,"thread_start_us":867,"threads_started":1}
I20260806 22:34:33.317487 26854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.297576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40072 (local address 127.15.198.131:44001)
0806 22:34:33.304419 (+ 6843us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:33.304435 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:33.304491 (+ 56us) server_negotiation.cc:407] Connection header received
0806 22:34:33.304655 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:33.304670 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:33.305112 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:33.305448 (+ 336us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:33.306640 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.311239 (+ 4599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.314076 (+ 2837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.315908 (+ 1832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.316094 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.316298 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:33.316907 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 22:34:33.317108 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6516,"thread_start_us":4354,"threads_started":1}
I20260806 22:34:33.317548 26853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.297254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:44001 (local address 127.0.0.1:40072)
0806 22:34:33.298113 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:33.298141 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:33.298380 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:33.305413 (+ 7033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:33.305417 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:33.305451 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:33.306252 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:33.306267 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.311492 (+ 5225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.311498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:33.313822 (+ 2324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:33.313837 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.316487 (+ 2650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.316494 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:33.316677 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.316706 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 22:34:33.316825 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:34:33.316998 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":30,"thread_start_us":388,"threads_started":1}
I20260806 22:34:33.321244 26487 tablet_service.cc:1467] Processing CreateTablet for tablet 22c6ff7ecade4678840e20fd3f721307 (DEFAULT_TABLE table=test-table [id=c24addbe5331424f80578970f73cb6d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:33.322593 26855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.298862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:45833 (local address 127.0.0.1:42798)
0806 22:34:33.310523 (+ 11661us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:33.310549 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:33.310698 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:33.312664 (+ 1966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:33.312670 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:33.312702 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:33.313620 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:33.313633 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.316471 (+ 2838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.316479 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:33.319222 (+ 2743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:33.319244 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.321523 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.321530 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:33.321733 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.321773 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 22:34:33.321886 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 22:34:33.322073 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11509,"mutex_wait_us":291,"thread_start_us":676,"threads_started":1}
I20260806 22:34:33.322968 26858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.299253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42798 (local address 127.15.198.129:45833)
0806 22:34:33.311747 (+ 12494us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:33.311761 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:33.311802 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:34:33.311963 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:33.311973 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:33.312356 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:33.312812 (+ 456us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:33.313881 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.316286 (+ 2405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.319439 (+ 3153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.321264 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.321475 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.321692 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:33.321974 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 22:34:33.322168 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11845,"thread_start_us":12007,"threads_started":1}
I20260806 22:34:33.322477 26716 tablet_service.cc:1467] Processing CreateTablet for tablet 22c6ff7ecade4678840e20fd3f721307 (DEFAULT_TABLE table=test-table [id=c24addbe5331424f80578970f73cb6d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:33.323431 26487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22c6ff7ecade4678840e20fd3f721307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:33.324680 26716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22c6ff7ecade4678840e20fd3f721307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:33.328198 26285 tablet_service.cc:1467] Processing CreateTablet for tablet 22c6ff7ecade4678840e20fd3f721307 (DEFAULT_TABLE table=test-table [id=c24addbe5331424f80578970f73cb6d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:33.330132 26285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22c6ff7ecade4678840e20fd3f721307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:33.341524 26859 tablet_bootstrap.cc:492] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03: Bootstrap starting.
I20260806 22:34:33.344475 26861 tablet_bootstrap.cc:492] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c: Bootstrap starting.
I20260806 22:34:33.347318 26859 tablet_bootstrap.cc:654] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:33.349112 26861 tablet_bootstrap.cc:654] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:33.349102 26859 log.cc:824] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:33.351244 26861 log.cc:824] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:33.352039 26863 tablet_bootstrap.cc:492] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8: Bootstrap starting.
I20260806 22:34:33.356998 26863 tablet_bootstrap.cc:654] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:33.358484 26863 log.cc:824] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:33.358724 26861 tablet_bootstrap.cc:492] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c: No bootstrap required, opened a new log
I20260806 22:34:33.358728 26859 tablet_bootstrap.cc:492] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03: No bootstrap required, opened a new log
I20260806 22:34:33.359138 26861 ts_tablet_manager.cc:1397] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c: Time spent bootstrapping tablet: real 0.015s user 0.014s sys 0.000s
I20260806 22:34:33.359179 26859 ts_tablet_manager.cc:1397] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.011s
I20260806 22:34:33.362056 26863 tablet_bootstrap.cc:492] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8: No bootstrap required, opened a new log
I20260806 22:34:33.362452 26863 ts_tablet_manager.cc:1397] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.007s
I20260806 22:34:33.372732 26859 raft_consensus.cc:348] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:33.372738 26861 raft_consensus.cc:348] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:33.373395 26861 raft_consensus.cc:374] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:33.373382 26859 raft_consensus.cc:374] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:33.373662 26859 raft_consensus.cc:729] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f8024a031c4473aaff990b586743b03, State: Initialized, Role: FOLLOWER
I20260806 22:34:33.373674 26861 raft_consensus.cc:729] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7138c95721d2478f946f0a793fb57b8c, State: Initialized, Role: FOLLOWER
I20260806 22:34:33.373473 26863 raft_consensus.cc:348] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:33.374049 26863 raft_consensus.cc:374] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:33.374193 26863 raft_consensus.cc:729] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4630b0558449a09775c2ad06c9daf8, State: Initialized, Role: FOLLOWER
I20260806 22:34:33.374799 26859 consensus_queue.cc:260] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:33.374742 26861 consensus_queue.cc:260] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:33.374974 26863 consensus_queue.cc:260] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:33.384172 26859 ts_tablet_manager.cc:1428] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03: Time spent starting tablet: real 0.025s user 0.018s sys 0.005s
I20260806 22:34:33.384178 26861 ts_tablet_manager.cc:1428] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c: Time spent starting tablet: real 0.025s user 0.018s sys 0.005s
I20260806 22:34:33.386142 26863 ts_tablet_manager.cc:1428] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8: Time spent starting tablet: real 0.024s user 0.014s sys 0.008s
I20260806 22:34:33.386696 26602 heartbeater.cc:503] Master 127.15.198.190:38077 was elected leader, sending a full tablet report...
I20260806 22:34:33.388530 26805 heartbeater.cc:503] Master 127.15.198.190:38077 was elected leader, sending a full tablet report...
W20260806 22:34:33.415527 26808 tablet.cc:2369] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:34:33.418840 26867 raft_consensus.cc:482] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:34:33.419286 26867 raft_consensus.cc:504] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:33.421485 26867 leader_election.cc:288] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7138c95721d2478f946f0a793fb57b8c (127.15.198.130:40791), cc4630b0558449a09775c2ad06c9daf8 (127.15.198.129:45833)
I20260806 22:34:33.429126 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.422063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:38991 (local address 127.15.198.130:40791)
0806 22:34:33.422318 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:33.422325 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:33.422573 (+ 248us) server_negotiation.cc:407] Connection header received
0806 22:34:33.422846 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:33.422857 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:33.422932 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:33.423059 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:33.423558 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.424377 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.426988 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.427981 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.428162 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.428259 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:33.428603 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 22:34:33.428764 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 22:34:33.429234 26858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.422090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:56591 (local address 127.15.198.129:45833)
0806 22:34:33.422318 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:33.422325 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:33.422580 (+ 255us) server_negotiation.cc:407] Connection header received
0806 22:34:33.422857 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:33.422862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:33.422937 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:33.423100 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:33.423564 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.424414 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.426981 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.428081 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.428282 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.428372 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:33.428727 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 22:34:33.428883 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 22:34:33.429641 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.422255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:40791 (local address 127.15.198.131:38991)
0806 22:34:33.422457 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:33.422473 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:33.422681 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:33.423089 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:33.423094 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:33.423118 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:33.423402 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:33.423412 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.424516 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.424522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:33.426768 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:33.426784 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.428150 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.428156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:33.428424 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.428453 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 22:34:33.428572 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:34:33.428748 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":29,"spinlock_wait_cycles":15744}
I20260806 22:34:33.429790 26821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.421928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:45833 (local address 127.15.198.131:56591)
0806 22:34:33.422434 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:33.422472 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:33.422682 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:33.423117 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:33.423122 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:33.423147 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:33.423408 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:33.423417 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.424738 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.424745 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:33.426773 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:33.426787 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.428295 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.428302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:33.428535 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.428561 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 22:34:33.428695 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 22:34:33.428848 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":21,"spinlock_wait_cycles":20480}
I20260806 22:34:33.431084 26522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22c6ff7ecade4678840e20fd3f721307" candidate_uuid: "8f8024a031c4473aaff990b586743b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7138c95721d2478f946f0a793fb57b8c" is_pre_election: true
I20260806 22:34:33.431043 26313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22c6ff7ecade4678840e20fd3f721307" candidate_uuid: "8f8024a031c4473aaff990b586743b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4630b0558449a09775c2ad06c9daf8" is_pre_election: true
I20260806 22:34:33.431819 26522 raft_consensus.cc:2454] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f8024a031c4473aaff990b586743b03 in term 0.
I20260806 22:34:33.431936 26313 raft_consensus.cc:2454] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f8024a031c4473aaff990b586743b03 in term 0.
I20260806 22:34:33.432909 26641 leader_election.cc:302] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f8024a031c4473aaff990b586743b03, cc4630b0558449a09775c2ad06c9daf8; no voters:
I20260806 22:34:33.433627 26867 raft_consensus.cc:2781] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:34:33.433975 26867 raft_consensus.cc:482] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:34:33.434118 26867 raft_consensus.cc:3037] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:33.437294 26867 raft_consensus.cc:504] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:33.438060 26867 leader_election.cc:288] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [CANDIDATE]: Term 1 election: Requested vote from peers 7138c95721d2478f946f0a793fb57b8c (127.15.198.130:40791), cc4630b0558449a09775c2ad06c9daf8 (127.15.198.129:45833)
I20260806 22:34:33.438807 26313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22c6ff7ecade4678840e20fd3f721307" candidate_uuid: "8f8024a031c4473aaff990b586743b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4630b0558449a09775c2ad06c9daf8"
I20260806 22:34:33.439131 26313 raft_consensus.cc:3037] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:33.438994 26522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22c6ff7ecade4678840e20fd3f721307" candidate_uuid: "8f8024a031c4473aaff990b586743b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7138c95721d2478f946f0a793fb57b8c"
I20260806 22:34:33.439464 26522 raft_consensus.cc:3037] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:33.442662 26522 raft_consensus.cc:2454] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f8024a031c4473aaff990b586743b03 in term 1.
I20260806 22:34:33.443404 26313 raft_consensus.cc:2454] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f8024a031c4473aaff990b586743b03 in term 1.
I20260806 22:34:33.444125 26641 leader_election.cc:302] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f8024a031c4473aaff990b586743b03, cc4630b0558449a09775c2ad06c9daf8; no voters:
I20260806 22:34:33.444624 26867 raft_consensus.cc:2781] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:33.451489 26867 raft_consensus.cc:686] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [term 1 LEADER]: Becoming Leader. State: Replica: 8f8024a031c4473aaff990b586743b03, State: Running, Role: LEADER
I20260806 22:34:33.452862 26867 consensus_queue.cc:237] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:33.460824 24953 catalog_manager.cc:5495] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f8024a031c4473aaff990b586743b03 (127.15.198.131). New cstate: current_term: 1 leader_uuid: "8f8024a031c4473aaff990b586743b03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } health_report { overall_health: UNKNOWN } } }
I20260806 22:34:33.513729 26854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.506133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40074 (local address 127.15.198.131:44001)
0806 22:34:33.506443 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:33.506451 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:33.506479 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:34:33.506655 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:33.506660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:33.506769 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:33.506976 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:34:33.507497 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.508585 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.510424 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.511236 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.511402 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.512168 (+ 766us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:34:33.512933 (+ 765us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:34:33.513169 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:33.513246 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:34:33.513366 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 22:34:33.525763 26522 raft_consensus.cc:1261] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c [term 1 FOLLOWER]: Refusing update from remote peer 8f8024a031c4473aaff990b586743b03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:33.525763 26313 raft_consensus.cc:1261] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8 [term 1 FOLLOWER]: Refusing update from remote peer 8f8024a031c4473aaff990b586743b03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:33.527171 26870 consensus_queue.cc:1035] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:33.527752 26867 consensus_queue.cc:1035] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:33.560397 26878 mvcc.cc:205] Tried to move back new op lower bound from 7315684038746640384 to 7315684038468595712. Current Snapshot: MvccSnapshot[applied={T|T < 7315684038746640384}]
I20260806 22:34:33.572149 24953 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:34:33.572638 24953 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:34:33.583269 24953 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27fab5d8c68046e3973078cbdc8d5841: Sending DeleteTablet for 3 replicas of tablet 22c6ff7ecade4678840e20fd3f721307
I20260806 22:34:33.584822 26716 tablet_service.cc:1514] Processing DeleteTablet for tablet 22c6ff7ecade4678840e20fd3f721307 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:33 UTC) from {username='oryx', principal='oryx/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:40072
I20260806 22:34:33.584944 26487 tablet_service.cc:1514] Processing DeleteTablet for tablet 22c6ff7ecade4678840e20fd3f721307 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:33 UTC) from {username='oryx', principal='oryx/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:53572
I20260806 22:34:33.585240 26285 tablet_service.cc:1514] Processing DeleteTablet for tablet 22c6ff7ecade4678840e20fd3f721307 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:33 UTC) from {username='oryx', principal='oryx/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:42798
I20260806 22:34:33.610881 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 24905
I20260806 22:34:33.612876 25000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:33.621481 26891 tablet_replica.cc:330] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03: stopping tablet replica
I20260806 22:34:33.622673 26891 raft_consensus.cc:2229] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:34:33.623960 26891 raft_consensus.cc:2258] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:33.630383 26892 tablet_replica.cc:330] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c: stopping tablet replica
I20260806 22:34:33.631835 26894 tablet_replica.cc:330] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8: stopping tablet replica
I20260806 22:34:33.632675 26891 ts_tablet_manager.cc:1905] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:33.633376 26892 raft_consensus.cc:2229] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:34:33.634058 26894 raft_consensus.cc:2229] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:34:33.639701 26892 raft_consensus.cc:2258] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:33.640425 26894 raft_consensus.cc:2258] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:33.644646 26834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.588929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45170 (local address 127.15.198.189:44373)
0806 22:34:33.589173 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:33.589179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:33.596010 (+ 6831us) server_negotiation.cc:407] Connection header received
0806 22:34:33.596314 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:33.596321 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:33.596451 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:33.596566 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:33.608291 (+ 11725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.609560 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.627786 (+ 18226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.628755 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.628903 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.634225 (+ 5322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:33.634235 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:33.634240 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:33.634301 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:33.639195 (+ 4894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.640565 (+ 1370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.640573 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.640577 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.640820 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.641699 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.641709 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.641715 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.642490 (+ 775us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:33.642907 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:33.643791 (+ 884us) server_negotiation.cc:299] Negotiation successful
0806 22:34:33.644135 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 22:34:33.648242 26894 ts_tablet_manager.cc:1905] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:33.650987 26892 ts_tablet_manager.cc:1905] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:33.672111 26839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.595699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40770 (local address 127.15.198.188:37991)
0806 22:34:33.595939 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:33.595946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:33.647853 (+ 51907us) server_negotiation.cc:407] Connection header received
0806 22:34:33.648182 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:33.648189 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:33.648310 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:33.648398 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:33.651322 (+ 2924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.652773 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.655462 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.656321 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.656462 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.661860 (+ 5398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:33.661871 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:33.661877 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:33.661946 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:33.667257 (+ 5311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.668637 (+ 1380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.668646 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.668650 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.668846 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.669373 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.669379 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.669383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.669896 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:33.670379 (+ 483us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:33.671315 (+ 936us) server_negotiation.cc:299] Negotiation successful
0806 22:34:33.671682 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 22:34:33.679766 26894 ts_tablet_manager.cc:1918] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:34:33.680068 26894 log.cc:1196] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/22c6ff7ecade4678840e20fd3f721307
I20260806 22:34:33.681085 26894 ts_tablet_manager.cc:1939] T 22c6ff7ecade4678840e20fd3f721307 P cc4630b0558449a09775c2ad06c9daf8: Deleting consensus metadata
I20260806 22:34:33.681340 26891 ts_tablet_manager.cc:1918] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:34:33.681711 26891 log.cc:1196] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/22c6ff7ecade4678840e20fd3f721307
I20260806 22:34:33.682597 26891 ts_tablet_manager.cc:1939] T 22c6ff7ecade4678840e20fd3f721307 P 8f8024a031c4473aaff990b586743b03: Deleting consensus metadata
I20260806 22:34:33.686391 26892 ts_tablet_manager.cc:1918] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:34:33.686715 26892 log.cc:1196] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/22c6ff7ecade4678840e20fd3f721307
I20260806 22:34:33.687825 26892 ts_tablet_manager.cc:1939] T 22c6ff7ecade4678840e20fd3f721307 P 7138c95721d2478f946f0a793fb57b8c: Deleting consensus metadata
I20260806 22:34:33.792834 24931 catalog_manager.cc:4841] TS cc4630b0558449a09775c2ad06c9daf8 (127.15.198.129:45833): tablet 22c6ff7ecade4678840e20fd3f721307 (table test-table [id=c24addbe5331424f80578970f73cb6d3]) successfully deleted
I20260806 22:34:33.793974 24933 catalog_manager.cc:4841] TS 7138c95721d2478f946f0a793fb57b8c (127.15.198.130:40791): tablet 22c6ff7ecade4678840e20fd3f721307 (table test-table [id=c24addbe5331424f80578970f73cb6d3]) successfully deleted
I20260806 22:34:33.794559 24932 catalog_manager.cc:4841] TS 8f8024a031c4473aaff990b586743b03 (127.15.198.131:44001): tablet 22c6ff7ecade4678840e20fd3f721307 (table test-table [id=c24addbe5331424f80578970f73cb6d3]) successfully deleted
I20260806 22:34:33.795356 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 24905
I20260806 22:34:33.883323 26839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.851912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40784 (local address 127.15.198.188:37991)
0806 22:34:33.852168 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:33.852174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:33.864456 (+ 12282us) server_negotiation.cc:407] Connection header received
0806 22:34:33.864701 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:33.864708 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:33.864813 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:33.864927 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:33.867900 (+ 2973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.869200 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.871134 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.871846 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.871994 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.875651 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:33.875657 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:33.875660 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:33.875700 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:33.879654 (+ 3954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.880693 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.880699 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.880703 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.880866 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.881344 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.881350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.881354 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.881859 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:33.882170 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:33.882562 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 22:34:33.882854 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 22:34:33.883523 26834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:33.845364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45176 (local address 127.15.198.189:44373)
0806 22:34:33.851262 (+ 5898us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:33.851271 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:33.856367 (+ 5096us) server_negotiation.cc:407] Connection header received
0806 22:34:33.856740 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:33.856749 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:33.856873 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:33.857026 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:33.867451 (+ 10425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.868639 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.870808 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:33.872719 (+ 1911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:33.872856 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:33.876277 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:33.876285 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:33.876292 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:33.876350 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:33.880265 (+ 3915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.881113 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.881119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.881123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.881311 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:33.881698 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:33.881703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:33.881707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:33.882097 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:33.882373 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:33.882919 (+ 546us) server_negotiation.cc:299] Negotiation successful
0806 22:34:33.883225 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5683}
I20260806 22:34:34.560897 26907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:34.559870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.129:40177)
0806 22:34:34.560523 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:34.560678 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":352,"threads_started":1}
W20260806 22:34:34.561602 26376 heartbeater.cc:644] Failed to heartbeat to 127.15.198.190:38077 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
I20260806 22:34:34.565526 26908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:34.564508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.131:46803)
0806 22:34:34.565065 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:34.565240 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":293,"threads_started":1}
W20260806 22:34:34.566009 26805 heartbeater.cc:644] Failed to heartbeat to 127.15.198.190:38077 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
I20260806 22:34:34.569343 26909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:34.568081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.130:46413)
0806 22:34:34.568856 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:34.569031 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":501,"threads_started":1}
W20260806 22:34:34.569813 26602 heartbeater.cc:644] Failed to heartbeat to 127.15.198.190:38077 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
I20260806 22:34:35.083197 26910 raft_consensus.cc:482] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27fab5d8c68046e3973078cbdc8d5841)
I20260806 22:34:35.083453 26913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27fab5d8c68046e3973078cbdc8d5841)
I20260806 22:34:35.083528 26910 raft_consensus.cc:504] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false } }
I20260806 22:34:35.083779 26913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false } }
I20260806 22:34:35.084838 26913 leader_election.cc:288] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27fab5d8c68046e3973078cbdc8d5841 (127.15.198.190:38077), b486dba97ea24ba7804d250a73a92c3b (127.15.198.189:44373)
I20260806 22:34:35.090438 26910 leader_election.cc:288] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27fab5d8c68046e3973078cbdc8d5841 (127.15.198.190:38077), 5dd49501d6894cc49985c8633b52ff67 (127.15.198.188:37991)
I20260806 22:34:35.099612 26919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.090015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:54260)
0806 22:34:35.098649 (+ 8634us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.099067 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8461,"thread_start_us":8324,"threads_started":1}
I20260806 22:34:35.100445 26915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.084972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:54246)
0806 22:34:35.098907 (+ 13935us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.100034 (+ 1127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13759,"mutex_wait_us":107,"thread_start_us":13160,"threads_started":1}
W20260806 22:34:35.100852 25799 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
W20260806 22:34:35.100857 26075 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
I20260806 22:34:35.103672 26915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.103092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:54276)
0806 22:34:35.103354 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.103474 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260806 22:34:35.104038 25799 leader_election.cc:334] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 27fab5d8c68046e3973078cbdc8d5841 (127.15.198.190:38077): Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
I20260806 22:34:35.104899 26919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.104417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:54290)
0806 22:34:35.104595 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.104711 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 22:34:35.105849 26075 leader_election.cc:334] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 27fab5d8c68046e3973078cbdc8d5841 (127.15.198.190:38077): Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
I20260806 22:34:35.107028 26917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.085096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:44373 (local address 127.0.0.1:45178)
0806 22:34:35.098002 (+ 12906us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.098036 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:35.098506 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:35.099361 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:35.099366 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:35.099401 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:35.099826 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:35.099845 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.101832 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.101838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:35.104417 (+ 2579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:35.104439 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.106009 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.106016 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:35.106213 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.106242 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 22:34:35.106361 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:34:35.106544 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12637,"mutex_wait_us":4,"thread_start_us":12448,"threads_started":1}
I20260806 22:34:35.107038 26918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.085306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45178 (local address 127.15.198.189:44373)
0806 22:34:35.098010 (+ 12704us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:35.098019 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:35.098317 (+ 298us) server_negotiation.cc:407] Connection header received
0806 22:34:35.098706 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:35.098713 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:35.098815 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:35.098932 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:35.100173 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.101608 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.104654 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.105807 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.106009 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.106099 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:35.106410 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 22:34:35.106561 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":12418,"thread_start_us":12113,"threads_started":1}
I20260806 22:34:35.107790 25829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5dd49501d6894cc49985c8633b52ff67" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b486dba97ea24ba7804d250a73a92c3b" is_pre_election: true
I20260806 22:34:35.108142 25829 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd49501d6894cc49985c8633b52ff67 in term 1.
I20260806 22:34:35.108723 26072 leader_election.cc:302] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5dd49501d6894cc49985c8633b52ff67, b486dba97ea24ba7804d250a73a92c3b; no voters: 27fab5d8c68046e3973078cbdc8d5841
I20260806 22:34:35.109140 26913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 22:34:35.109328 26913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 27fab5d8c68046e3973078cbdc8d5841)
I20260806 22:34:35.109496 26913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:34:35.109532 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.084972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37991 (local address 127.0.0.1:40796)
0806 22:34:35.098208 (+ 13236us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.098233 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:35.098422 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:35.100935 (+ 2513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:35.100941 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:35.100976 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:35.101623 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:35.101635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.104841 (+ 3206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.104852 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:35.106782 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:35.106796 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.108558 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.108565 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:35.108750 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.108776 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 22:34:35.108906 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 22:34:35.109077 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13093,"mutex_wait_us":64,"thread_start_us":13086,"threads_started":1}
I20260806 22:34:35.109618 26920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.098806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40796 (local address 127.15.198.188:37991)
0806 22:34:35.099681 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:35.099688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:35.099707 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:34:35.100446 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:35.100451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:35.100530 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:35.100641 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:35.101832 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.104702 (+ 2870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.106967 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.108309 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.108511 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.108607 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:35.108945 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 22:34:35.109095 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":464,"server-negotiator.queue_time_us":736,"thread_start_us":504,"threads_started":1}
I20260806 22:34:35.110468 26110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b486dba97ea24ba7804d250a73a92c3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5dd49501d6894cc49985c8633b52ff67" is_pre_election: true
I20260806 22:34:35.112491 26913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false } }
I20260806 22:34:35.113101 26110 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b486dba97ea24ba7804d250a73a92c3b in current term 2: Already voted for candidate 5dd49501d6894cc49985c8633b52ff67 in this term.
I20260806 22:34:35.113454 26913 leader_election.cc:288] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [CANDIDATE]: Term 2 election: Requested vote from peers 27fab5d8c68046e3973078cbdc8d5841 (127.15.198.190:38077), b486dba97ea24ba7804d250a73a92c3b (127.15.198.189:44373)
I20260806 22:34:35.114089 26917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.113616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:54302)
0806 22:34:35.113791 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.113915 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 22:34:35.114168 25798 leader_election.cc:302] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b486dba97ea24ba7804d250a73a92c3b; no voters: 27fab5d8c68046e3973078cbdc8d5841, 5dd49501d6894cc49985c8633b52ff67
I20260806 22:34:35.114225 25829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5dd49501d6894cc49985c8633b52ff67" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b486dba97ea24ba7804d250a73a92c3b"
I20260806 22:34:35.114547 25829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:34:35.115837 26917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.115407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:54314)
0806 22:34:35.115582 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.115688 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 22:34:35.116031 26075 leader_election.cc:334] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 27fab5d8c68046e3973078cbdc8d5841 (127.15.198.190:38077): Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
I20260806 22:34:35.117499 25829 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd49501d6894cc49985c8633b52ff67 in term 2.
I20260806 22:34:35.117743 26910 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:34:35.118083 26072 leader_election.cc:302] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5dd49501d6894cc49985c8633b52ff67, b486dba97ea24ba7804d250a73a92c3b; no voters: 27fab5d8c68046e3973078cbdc8d5841
I20260806 22:34:35.118458 26913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:34:35.119048 26913 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [term 2 LEADER]: Becoming Leader. State: Replica: 5dd49501d6894cc49985c8633b52ff67, State: Running, Role: LEADER
I20260806 22:34:35.119542 26913 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false } }
I20260806 22:34:35.121564 26926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dd49501d6894cc49985c8633b52ff67. Latest consensus state: current_term: 2 leader_uuid: "5dd49501d6894cc49985c8633b52ff67" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false } } }
I20260806 22:34:35.121932 26926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:35.122819 26929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:34:35.128114 26929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:34:35.129226 26929 catalog_manager.cc:1259] Loaded cluster ID: c13aa0eae9f240ceb2a6470f0e8d05bb
I20260806 22:34:35.129320 26929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:34:35.130712 26929 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:34:35.131790 26929 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: Loaded TSK: 0
I20260806 22:34:35.133019 26929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:34:35.158790 26094 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40784:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:34:35.167526 26094 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:34:35.173508 26917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.172933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:54336)
0806 22:34:35.173162 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.173314 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 22:34:35.173746 25829 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [term 2 FOLLOWER]: Refusing update from remote peer 5dd49501d6894cc49985c8633b52ff67: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 22:34:35.174779 26917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.174283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:54352)
0806 22:34:35.174482 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.174607 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 22:34:35.174759 26926 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 22:34:35.175247 26075 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 -> Peer 27fab5d8c68046e3973078cbdc8d5841 (127.15.198.190:38077): Couldn't send request to peer 27fab5d8c68046e3973078cbdc8d5841. Status: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 22:34:35.178299 26910 sys_catalog.cc:455] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dd49501d6894cc49985c8633b52ff67. Latest consensus state: current_term: 2 leader_uuid: "5dd49501d6894cc49985c8633b52ff67" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false } } }
I20260806 22:34:35.178649 26910 sys_catalog.cc:458] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:34:35.180524 26926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "5dd49501d6894cc49985c8633b52ff67" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false } } }
I20260806 22:34:35.180891 26926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:35.181955 26910 sys_catalog.cc:455] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "5dd49501d6894cc49985c8633b52ff67" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "27fab5d8c68046e3973078cbdc8d5841" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38077 } } peers { permanent_uuid: "b486dba97ea24ba7804d250a73a92c3b" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 44373 } attrs { promote: false } } peers { permanent_uuid: "5dd49501d6894cc49985c8633b52ff67" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37991 } attrs { promote: false } } }
I20260806 22:34:35.182328 26910 sys_catalog.cc:458] T 00000000000000000000000000000000 P b486dba97ea24ba7804d250a73a92c3b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:34:35.200580 26917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.191811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:44001 (local address 127.0.0.1:40078)
0806 22:34:35.192004 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.192030 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:35.192188 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:35.193206 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:35.193211 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:35.193247 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:35.193547 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:35.193563 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.195432 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.195438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:35.196959 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:35.196978 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.199450 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.199457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:35.199682 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.199723 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:34:35.199833 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 22:34:35.200036 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 22:34:35.200676 26937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.192004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40078 (local address 127.15.198.131:44001)
0806 22:34:35.192743 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:35.192750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:35.192770 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:34:35.192891 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:35.192897 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:35.192997 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:35.193738 (+ 741us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:35.193986 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.195294 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.197195 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.199148 (+ 1953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.199574 (+ 426us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.199714 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:35.199870 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 22:34:35.200109 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":425,"threads_started":1}
I20260806 22:34:35.201582 26919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.192225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:40791 (local address 127.0.0.1:53576)
0806 22:34:35.192461 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.192481 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:35.192687 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:35.193917 (+ 1230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:35.193922 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:35.193962 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:35.194421 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:35.194439 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.196625 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.196631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:35.198602 (+ 1971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:35.198616 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.200801 (+ 2185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.200810 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:35.201001 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.201045 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:34:35.201116 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 22:34:35.201290 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 22:34:35.202244 26716 tablet_service.cc:1467] Processing CreateTablet for tablet 8a3c57078af74c41a2feaa7a113bdfee (DEFAULT_TABLE table=test-table [id=406d53faaeae48de97f02dca83d9a45a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:35.202791 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.192783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53576 (local address 127.15.198.130:40791)
0806 22:34:35.193427 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:35.193434 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:35.193456 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:34:35.193554 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:35.193558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:35.193665 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:35.194653 (+ 988us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:35.194882 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.196448 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.198873 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.200523 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.202018 (+ 1495us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.202196 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:35.202253 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 22:34:35.202430 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":352,"threads_started":1}
I20260806 22:34:35.202929 26716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a3c57078af74c41a2feaa7a113bdfee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:35.204152 26487 tablet_service.cc:1467] Processing CreateTablet for tablet 8a3c57078af74c41a2feaa7a113bdfee (DEFAULT_TABLE table=test-table [id=406d53faaeae48de97f02dca83d9a45a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:35.204701 26487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a3c57078af74c41a2feaa7a113bdfee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:35.212069 26944 tablet_bootstrap.cc:492] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03: Bootstrap starting.
I20260806 22:34:35.213894 26945 tablet_bootstrap.cc:492] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c: Bootstrap starting.
I20260806 22:34:35.215278 26944 tablet_bootstrap.cc:654] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:35.217844 26945 tablet_bootstrap.cc:654] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:35.218345 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.194522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42806 (local address 127.15.198.129:45833)
0806 22:34:35.210395 (+ 15873us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:35.210402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:35.210423 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:34:35.210516 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:35.210520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:35.210622 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:35.210784 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:35.211545 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.212975 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.215555 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.216974 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.217171 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.217269 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:35.217747 (+ 478us) server_negotiation.cc:299] Negotiation successful
0806 22:34:35.217926 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15721,"thread_start_us":16479,"threads_started":1}
I20260806 22:34:35.218349 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.193918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:45833 (local address 127.0.0.1:42806)
0806 22:34:35.194654 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.194674 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:35.194798 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:35.210807 (+ 16009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:35.210814 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:35.210851 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:35.211327 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:35.211342 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.213181 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.213192 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:35.215332 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:35.215367 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.217256 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.217266 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:35.217503 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.217546 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:34:35.217707 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 22:34:35.217899 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":817,"threads_started":1}
I20260806 22:34:35.218410 26944 tablet_bootstrap.cc:492] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03: No bootstrap required, opened a new log
I20260806 22:34:35.218613 26944 ts_tablet_manager.cc:1397] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.003s
I20260806 22:34:35.219620 26944 raft_consensus.cc:348] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:35.219667 26285 tablet_service.cc:1467] Processing CreateTablet for tablet 8a3c57078af74c41a2feaa7a113bdfee (DEFAULT_TABLE table=test-table [id=406d53faaeae48de97f02dca83d9a45a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:35.219897 26944 raft_consensus.cc:374] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:35.220012 26944 raft_consensus.cc:729] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f8024a031c4473aaff990b586743b03, State: Initialized, Role: FOLLOWER
I20260806 22:34:35.220238 26285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a3c57078af74c41a2feaa7a113bdfee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:35.220372 26944 consensus_queue.cc:260] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:35.220688 26945 tablet_bootstrap.cc:492] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c: No bootstrap required, opened a new log
I20260806 22:34:35.220894 26945 ts_tablet_manager.cc:1397] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260806 22:34:35.222144 26945 raft_consensus.cc:348] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:35.222515 26945 raft_consensus.cc:374] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:35.222628 26945 raft_consensus.cc:729] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7138c95721d2478f946f0a793fb57b8c, State: Initialized, Role: FOLLOWER
I20260806 22:34:35.223089 26945 consensus_queue.cc:260] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:35.226336 26953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.225117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.130:39367)
0806 22:34:35.225793 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.226015 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":515,"threads_started":1}
I20260806 22:34:35.228083 26955 tablet_bootstrap.cc:492] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8: Bootstrap starting.
I20260806 22:34:35.230747 26955 tablet_bootstrap.cc:654] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:35.230872 26952 raft_consensus.cc:482] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:34:35.231462 26945 ts_tablet_manager.cc:1428] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260806 22:34:35.231169 26952 raft_consensus.cc:504] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:35.231837 26601 heartbeater.cc:503] Master 127.15.198.188:37991 was elected leader, sending a full tablet report...
I20260806 22:34:35.234149 26952 leader_election.cc:288] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f8024a031c4473aaff990b586743b03 (127.15.198.131:44001), cc4630b0558449a09775c2ad06c9daf8 (127.15.198.129:45833)
I20260806 22:34:35.247254 26944 ts_tablet_manager.cc:1428] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03: Time spent starting tablet: real 0.028s user 0.008s sys 0.010s
I20260806 22:34:35.247393 26953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.234819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:45833 (local address 127.15.198.130:53031)
0806 22:34:35.235337 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.235386 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:35.235623 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:35.237427 (+ 1804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:35.237447 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:35.237486 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:35.237960 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:35.237976 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.239997 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.240006 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:35.243334 (+ 3328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:35.243359 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.245975 (+ 2616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.245986 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:35.246358 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.246419 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 22:34:35.246547 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 22:34:35.246813 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":67}
I20260806 22:34:35.248353 26953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.236569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.130:41107)
0806 22:34:35.247859 (+ 11290us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.248056 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11129,"thread_start_us":9725,"threads_started":1}
I20260806 22:34:35.249287 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.236020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:53031 (local address 127.15.198.129:45833)
0806 22:34:35.236728 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:35.236737 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:35.236765 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:34:35.236913 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:35.236922 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:35.237025 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:35.237133 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:35.238186 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.239791 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.243636 (+ 3845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.245615 (+ 1979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.245821 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.245965 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:35.248589 (+ 2624us) server_negotiation.cc:299] Negotiation successful
0806 22:34:35.248870 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477}
I20260806 22:34:35.250121 26313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a3c57078af74c41a2feaa7a113bdfee" candidate_uuid: "7138c95721d2478f946f0a793fb57b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4630b0558449a09775c2ad06c9daf8" is_pre_election: true
I20260806 22:34:35.250519 26804 heartbeater.cc:503] Master 127.15.198.188:37991 was elected leader, sending a full tablet report...
W20260806 22:34:35.251639 26418 leader_election.cc:341] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc4630b0558449a09775c2ad06c9daf8 (127.15.198.129:45833): Illegal state: must be running to vote when last-logged opid is not known
I20260806 22:34:35.257432 26937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.235485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:49443 (local address 127.15.198.131:44001)
0806 22:34:35.235728 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:35.235735 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:35.235854 (+ 119us) server_negotiation.cc:407] Connection header received
0806 22:34:35.236461 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:35.236469 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:35.236571 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:35.236646 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:35.238771 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.241002 (+ 2231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.253426 (+ 12424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.255452 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.255647 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.255770 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:35.256695 (+ 925us) server_negotiation.cc:299] Negotiation successful
0806 22:34:35.256957 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 22:34:35.258662 26959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.234861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:44001 (local address 127.15.198.130:49443)
0806 22:34:35.235749 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.235770 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:35.236101 (+ 331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:35.237031 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:35.237037 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:35.237070 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:35.237668 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:35.237686 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.241400 (+ 3714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.241408 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:35.253105 (+ 11697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:35.253141 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.256085 (+ 2944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.256096 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:35.256415 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.256474 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 22:34:35.257752 (+ 1278us) client_negotiation.cc:240] Negotiation successful
0806 22:34:35.258042 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"mutex_wait_us":3842,"thread_start_us":1082,"threads_started":1}
I20260806 22:34:35.259874 26745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a3c57078af74c41a2feaa7a113bdfee" candidate_uuid: "7138c95721d2478f946f0a793fb57b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f8024a031c4473aaff990b586743b03" is_pre_election: true
I20260806 22:34:35.260466 26745 raft_consensus.cc:2454] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7138c95721d2478f946f0a793fb57b8c in term 0.
I20260806 22:34:35.261408 26419 leader_election.cc:302] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7138c95721d2478f946f0a793fb57b8c, 8f8024a031c4473aaff990b586743b03; no voters: cc4630b0558449a09775c2ad06c9daf8
I20260806 22:34:35.261658 26955 tablet_bootstrap.cc:492] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8: No bootstrap required, opened a new log
I20260806 22:34:35.261885 26955 ts_tablet_manager.cc:1397] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8: Time spent bootstrapping tablet: real 0.034s user 0.008s sys 0.021s
I20260806 22:34:35.262074 26952 raft_consensus.cc:2781] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:34:35.262235 26952 raft_consensus.cc:482] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:34:35.262357 26952 raft_consensus.cc:3037] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:35.263119 26955 raft_consensus.cc:348] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:35.263464 26955 raft_consensus.cc:374] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:35.263581 26955 raft_consensus.cc:729] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4630b0558449a09775c2ad06c9daf8, State: Initialized, Role: FOLLOWER
I20260806 22:34:35.263938 26955 consensus_queue.cc:260] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:35.264479 26961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.248034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.131:49775)
0806 22:34:35.263855 (+ 15821us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.264086 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15646,"thread_start_us":15266,"threads_started":1}
I20260806 22:34:35.266450 26952 raft_consensus.cc:504] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:35.267807 26952 leader_election.cc:288] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [CANDIDATE]: Term 1 election: Requested vote from peers 8f8024a031c4473aaff990b586743b03 (127.15.198.131:44001), cc4630b0558449a09775c2ad06c9daf8 (127.15.198.129:45833)
I20260806 22:34:35.268378 26313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a3c57078af74c41a2feaa7a113bdfee" candidate_uuid: "7138c95721d2478f946f0a793fb57b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4630b0558449a09775c2ad06c9daf8"
I20260806 22:34:35.268698 26313 raft_consensus.cc:3037] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:35.269980 26955 ts_tablet_manager.cc:1428] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260806 22:34:35.270824 26964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.265488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.129:41529)
0806 22:34:35.270232 (+ 4744us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.270471 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4473,"thread_start_us":4128,"threads_started":1}
I20260806 22:34:35.271365 26745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a3c57078af74c41a2feaa7a113bdfee" candidate_uuid: "7138c95721d2478f946f0a793fb57b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f8024a031c4473aaff990b586743b03"
I20260806 22:34:35.271653 26745 raft_consensus.cc:3037] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:35.272171 26964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.271806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.129:36405)
0806 22:34:35.271963 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.272047 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 22:34:35.274858 26745 raft_consensus.cc:2454] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7138c95721d2478f946f0a793fb57b8c in term 1.
I20260806 22:34:35.275511 26375 heartbeater.cc:503] Master 127.15.198.188:37991 was elected leader, sending a full tablet report...
I20260806 22:34:35.275835 26419 leader_election.cc:302] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7138c95721d2478f946f0a793fb57b8c, 8f8024a031c4473aaff990b586743b03; no voters:
I20260806 22:34:35.276279 26952 raft_consensus.cc:2781] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:35.276577 26313 raft_consensus.cc:2454] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7138c95721d2478f946f0a793fb57b8c in term 1.
I20260806 22:34:35.277467 26961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.266364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.131:51939)
0806 22:34:35.277116 (+ 10752us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.277245 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10623,"threads_started":1}
I20260806 22:34:35.277951 26952 raft_consensus.cc:686] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [term 1 LEADER]: Becoming Leader. State: Replica: 7138c95721d2478f946f0a793fb57b8c, State: Running, Role: LEADER
I20260806 22:34:35.278403 26952 consensus_queue.cc:237] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } }
I20260806 22:34:35.280313 26959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.279880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.130:49277)
0806 22:34:35.280042 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.280160 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 22:34:35.283650 26094 catalog_manager.cc:5495] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7138c95721d2478f946f0a793fb57b8c (127.15.198.130). New cstate: current_term: 1 leader_uuid: "7138c95721d2478f946f0a793fb57b8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7138c95721d2478f946f0a793fb57b8c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 40791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 } health_report { overall_health: UNKNOWN } } }
I20260806 22:34:35.320397 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.314102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53582 (local address 127.15.198.130:40791)
0806 22:34:35.314370 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:35.314376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:35.314399 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:34:35.314583 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:35.314587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:35.314699 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:35.314863 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:34:35.315444 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.316278 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.317853 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.318353 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.318487 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.318958 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:34:35.319776 (+ 818us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:34:35.319903 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:35.320030 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 22:34:35.320126 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 22:34:35.325760 26313 raft_consensus.cc:1261] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8 [term 1 FOLLOWER]: Refusing update from remote peer 7138c95721d2478f946f0a793fb57b8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:35.326171 26745 raft_consensus.cc:1261] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03 [term 1 FOLLOWER]: Refusing update from remote peer 7138c95721d2478f946f0a793fb57b8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:35.326661 26966 consensus_queue.cc:1035] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4630b0558449a09775c2ad06c9daf8" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:35.327121 26952 consensus_queue.cc:1035] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f8024a031c4473aaff990b586743b03" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:35.330447 26964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.329561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.129:47021)
0806 22:34:35.329855 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.330026 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
I20260806 22:34:35.332456 26965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.331397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.131:33693)
0806 22:34:35.331790 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.332086 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260806 22:34:35.339169 26964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.338572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.129:56493)
0806 22:34:35.338823 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.338928 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260806 22:34:35.341285 26959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.340695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.130:36131)
0806 22:34:35.340960 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.341081 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260806 22:34:35.341871 26970 mvcc.cc:205] Tried to move back new op lower bound from 7315684046125408256 to 7315684045947969536. Current Snapshot: MvccSnapshot[applied={T|T < 7315684046125408256}]
I20260806 22:34:35.343540 26965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.342970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.15.198.131:54617)
0806 22:34:35.343211 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.343321 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260806 22:34:35.355899 26094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:34:35.356273 26094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:34:35.364208 26094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5dd49501d6894cc49985c8633b52ff67: Sending DeleteTablet for 3 replicas of tablet 8a3c57078af74c41a2feaa7a113bdfee
I20260806 22:34:35.365682 26487 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a3c57078af74c41a2feaa7a113bdfee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:35 UTC) from {username='oryx', principal='oryx/127.15.198.188@KRBTEST.COM'} at 127.0.0.1:53576
I20260806 22:34:35.366304 26716 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a3c57078af74c41a2feaa7a113bdfee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:35 UTC) from {username='oryx', principal='oryx/127.15.198.188@KRBTEST.COM'} at 127.0.0.1:40078
I20260806 22:34:35.366406 26285 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a3c57078af74c41a2feaa7a113bdfee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:35 UTC) from {username='oryx', principal='oryx/127.15.198.188@KRBTEST.COM'} at 127.0.0.1:42806
I20260806 22:34:35.368794 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 26047
I20260806 22:34:35.382539 26983 tablet_replica.cc:330] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8: stopping tablet replica
I20260806 22:34:35.382938 26983 raft_consensus.cc:2229] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:34:35.383589 26983 raft_consensus.cc:2258] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:35.385478 26983 ts_tablet_manager.cc:1905] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:35.389042 26981 tablet_replica.cc:330] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03: stopping tablet replica
I20260806 22:34:35.389330 26981 raft_consensus.cc:2229] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:34:35.389750 26981 raft_consensus.cc:2258] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:35.394696 26981 ts_tablet_manager.cc:1905] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:35.395234 26983 ts_tablet_manager.cc:1918] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:34:35.395457 26983 log.cc:1196] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/8a3c57078af74c41a2feaa7a113bdfee
I20260806 22:34:35.396179 26983 ts_tablet_manager.cc:1939] T 8a3c57078af74c41a2feaa7a113bdfee P cc4630b0558449a09775c2ad06c9daf8: Deleting consensus metadata
I20260806 22:34:35.398015 26074 catalog_manager.cc:4841] TS cc4630b0558449a09775c2ad06c9daf8 (127.15.198.129:45833): tablet 8a3c57078af74c41a2feaa7a113bdfee (table test-table [id=406d53faaeae48de97f02dca83d9a45a]) successfully deleted
I20260806 22:34:35.405597 26980 tablet_replica.cc:330] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c: stopping tablet replica
I20260806 22:34:35.405901 26980 raft_consensus.cc:2229] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:34:35.406927 26980 raft_consensus.cc:2258] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:35.408519 26980 ts_tablet_manager.cc:1905] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:35.409849 26981 ts_tablet_manager.cc:1918] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:34:35.410046 26981 log.cc:1196] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/8a3c57078af74c41a2feaa7a113bdfee
I20260806 22:34:35.410625 26981 ts_tablet_manager.cc:1939] T 8a3c57078af74c41a2feaa7a113bdfee P 8f8024a031c4473aaff990b586743b03: Deleting consensus metadata
I20260806 22:34:35.412348 26075 catalog_manager.cc:4841] TS 8f8024a031c4473aaff990b586743b03 (127.15.198.131:44001): tablet 8a3c57078af74c41a2feaa7a113bdfee (table test-table [id=406d53faaeae48de97f02dca83d9a45a]) successfully deleted
I20260806 22:34:35.417881 26980 ts_tablet_manager.cc:1918] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:34:35.418130 26980 log.cc:1196] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/8a3c57078af74c41a2feaa7a113bdfee
I20260806 22:34:35.418812 26980 ts_tablet_manager.cc:1939] T 8a3c57078af74c41a2feaa7a113bdfee P 7138c95721d2478f946f0a793fb57b8c: Deleting consensus metadata
I20260806 22:34:35.420499 26076 catalog_manager.cc:4841] TS 7138c95721d2478f946f0a793fb57b8c (127.15.198.130:40791): tablet 8a3c57078af74c41a2feaa7a113bdfee (table test-table [id=406d53faaeae48de97f02dca83d9a45a]) successfully deleted
I20260806 22:34:35.423301 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.370542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42810 (local address 127.15.198.129:45833)
0806 22:34:35.370835 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:35.370846 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:35.388066 (+ 17220us) server_negotiation.cc:407] Connection header received
0806 22:34:35.390434 (+ 2368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:35.390442 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:35.390580 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:35.390691 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:35.395416 (+ 4725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.396899 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.400219 (+ 3320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.401232 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.401402 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.413863 (+ 12461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:35.413911 (+ 48us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:35.413947 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:35.414016 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:35.419499 (+ 5483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:35.420686 (+ 1187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:35.420701 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:35.420705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:35.420941 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:35.421422 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:35.421426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:35.421429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:35.421874 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:35.422157 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:35.422578 (+ 421us) server_negotiation.cc:299] Negotiation successful
0806 22:34:35.422816 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 22:34:35.424307 26367 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:35.548287 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 26047
I20260806 22:34:35.587165 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 26393
I20260806 22:34:35.617005 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.588265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53592 (local address 127.15.198.130:40791)
0806 22:34:35.588566 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:35.588576 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:35.588656 (+ 80us) server_negotiation.cc:407] Connection header received
0806 22:34:35.588985 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:35.588992 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:35.589136 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:35.589273 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:35.591724 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.593427 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.595833 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.596680 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.596880 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.604702 (+ 7822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:35.604739 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:35.604771 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:35.604844 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:35.610791 (+ 5947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:35.612448 (+ 1657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:35.612466 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:35.612472 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:35.612842 (+ 370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:35.613455 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:35.613461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:35.613465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:35.614135 (+ 670us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:35.614552 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:35.616120 (+ 1568us) server_negotiation.cc:299] Negotiation successful
0806 22:34:35.616456 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 22:34:35.620581 26588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:35.624294 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.623686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:54366)
0806 22:34:35.623894 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.624009 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 22:34:35.625771 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.625273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:54374)
0806 22:34:35.625454 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:35.625582 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 22:34:35.771770 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 26393
I20260806 22:34:35.807878 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 26616
I20260806 22:34:35.825428 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:35.808890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40092 (local address 127.15.198.131:44001)
0806 22:34:35.809507 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:35.809512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:35.809531 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:34:35.809600 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:35.809604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:35.809677 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:35.809800 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:35.811453 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.812261 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.813690 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:35.814170 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:35.814284 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:35.817560 (+ 3276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:35.817583 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:35.817606 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:35.817661 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:35.821968 (+ 4307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:35.822751 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:35.822761 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:35.822764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:35.823089 (+ 325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:35.823472 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:35.823475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:35.823477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:35.824057 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:35.824362 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:35.824639 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 22:34:35.824938 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":296,"threads_started":1}
I20260806 22:34:35.826337 26797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:35.979882 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 26616
I20260806 22:34:36.018239 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 25621
I20260806 22:34:36.019708 25862 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:36.163697 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 25621
I20260806 22:34:36.167460 26994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:36.163831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:54380)
0806 22:34:36.166952 (+ 3121us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:36.167139 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2910,"thread_start_us":3089,"threads_started":1}
I20260806 22:34:36.171742 26994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:36.171297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38077 (local address 127.0.0.1:54386)
0806 22:34:36.171472 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:36.171567 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 22:34:36.199795 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 25777
I20260806 22:34:36.201215 26142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:36.347656 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 25777
2026-08-06T22:34:36Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (12360 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:34:36 dist-test-slave-h3ft krb5kdc[27000](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:34:36 dist-test-slave-h3ft krb5kdc[27000](info): set up 2 sockets
Aug 06 22:34:36 dist-test-slave-h3ft krb5kdc[27000](info): commencing operation
krb5kdc: starting...
W20260806 22:34:37.739967 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.336s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:34:37 dist-test-slave-h3ft krb5kdc[27000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:34:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:34:38Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.190.keytab.
Entry for principal oryx/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.190.keytab.
I20260806 22:34:38.105396 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:40953
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.190.keytab
--principal=oryx/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:36635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:40953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:38.552451 27020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:38.552980 27020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:38.553159 27020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:38.561506 27020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:34:38.561707 27020 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:34:38.561813 27020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:38.561952 27020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:38.562047 27020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:34:38.562155 27020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:34:38.573832 27020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:40953
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.190.keytab
--principal=oryx/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:40953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:38.575769 27020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:38.577579 27020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:38.592842 27025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:38.592824 27028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:38.592867 27026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:38.594166 27020 server_base.cc:1034] running on GCE node
I20260806 22:34:38.595388 27020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:38.597465 27020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:38.598773 27020 hybrid_clock.cc:648] HybridClock initialized: now 1786055678598652 us; error 90 us; skew 500 ppm
I20260806 22:34:38.599282 27020 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:38 dist-test-slave-h3ft krb5kdc[27000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055678, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:38.609021 27020 init.cc:377] Logged in from keytab as oryx/127.15.198.190@KRBTEST.COM (short username oryx)
I20260806 22:34:38.611681 27020 webserver.cc:460] Webserver started at http://127.15.198.190:44793/ using document root <none> and password file <none>
I20260806 22:34:38.612815 27020 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:38.613013 27020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:38.613492 27020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:38.617914 27020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "39467d156c884eadb5f658a1466a5745"
format_stamp: "Formatted at 2026-08-06 22:34:38 on dist-test-slave-h3ft"
server_key: "02fd1494f85b3c89912bf0b08c3d146c"
server_key_iv: "5a600addf07ddc7fe9e9f9a01059a2ed"
server_key_version: "encryptionkey@0"
I20260806 22:34:38.619190 27020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "39467d156c884eadb5f658a1466a5745"
format_stamp: "Formatted at 2026-08-06 22:34:38 on dist-test-slave-h3ft"
server_key: "02fd1494f85b3c89912bf0b08c3d146c"
server_key_iv: "5a600addf07ddc7fe9e9f9a01059a2ed"
server_key_version: "encryptionkey@0"
I20260806 22:34:38.627231 27020 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260806 22:34:38.632133 27035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:38.634151 27020 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:34:38.634369 27020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "39467d156c884eadb5f658a1466a5745"
format_stamp: "Formatted at 2026-08-06 22:34:38 on dist-test-slave-h3ft"
server_key: "02fd1494f85b3c89912bf0b08c3d146c"
server_key_iv: "5a600addf07ddc7fe9e9f9a01059a2ed"
server_key_version: "encryptionkey@0"
I20260806 22:34:38.634637 27020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:38.651765 27020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:38.657581 27020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:38.657891 27020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:38.686064 27020 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:40953
I20260806 22:34:38.686087 27118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:40953 every 8 connection(s)
I20260806 22:34:38.688221 27020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:34:38.692273 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 27020
I20260806 22:34:38.692631 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:34:38.693475 16154 external_mini_cluster.cc:1428] Setting key 28d73ebed27116a3bb01da9aa6173e46
I20260806 22:34:38.694170 27119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:38.713586 27119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745: Bootstrap starting.
Aug 06 22:34:38 dist-test-slave-h3ft krb5kdc[27000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM
I20260806 22:34:38.719266 27119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:38.721050 27119 log.cc:824] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:38.725872 27119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745: No bootstrap required, opened a new log
I20260806 22:34:38.726876 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:38.696335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60434 (local address 127.15.198.190:40953)
0806 22:34:38.697456 (+ 1121us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:38.697476 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:38.697545 (+ 69us) server_negotiation.cc:407] Connection header received
0806 22:34:38.699820 (+ 2275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:38.699865 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:38.700761 (+ 896us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:38.701465 (+ 704us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:38.703728 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:38.706449 (+ 2721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:38.708204 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:38.709464 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:38.709706 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:38.715292 (+ 5586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:38.715307 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:38.715333 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:38.715402 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:38.720412 (+ 5010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:38.721686 (+ 1274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:38.721702 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:38.721715 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:38.722065 (+ 350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:38.722670 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:38.722675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:38.722677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:38.723683 (+ 1006us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:38.724043 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:38.724562 (+ 519us) server_negotiation.cc:299] Negotiation successful
0806 22:34:38.725151 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":790,"thread_start_us":473,"threads_started":1}
I20260806 22:34:38.735982 27119 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39467d156c884eadb5f658a1466a5745" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40953 } }
I20260806 22:34:38.736737 27119 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:38.736909 27119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39467d156c884eadb5f658a1466a5745, State: Initialized, Role: FOLLOWER
I20260806 22:34:38.737692 27119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39467d156c884eadb5f658a1466a5745" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40953 } }
I20260806 22:34:38.737972 27119 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:34:38.738132 27119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:34:38.738351 27119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:38.741467 27119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39467d156c884eadb5f658a1466a5745" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40953 } }
I20260806 22:34:38.742151 27119 leader_election.cc:302] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39467d156c884eadb5f658a1466a5745; no voters:
I20260806 22:34:38.742928 27119 leader_election.cc:288] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:34:38.743191 27131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:38.744635 27131 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [term 1 LEADER]: Becoming Leader. State: Replica: 39467d156c884eadb5f658a1466a5745, State: Running, Role: LEADER
I20260806 22:34:38.745507 27131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39467d156c884eadb5f658a1466a5745" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40953 } }
I20260806 22:34:38.746043 27119 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:34:38.750248 27132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39467d156c884eadb5f658a1466a5745" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39467d156c884eadb5f658a1466a5745" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40953 } } }
I20260806 22:34:38.755505 27132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:38.751075 27134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39467d156c884eadb5f658a1466a5745. Latest consensus state: current_term: 1 leader_uuid: "39467d156c884eadb5f658a1466a5745" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39467d156c884eadb5f658a1466a5745" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40953 } } }
I20260806 22:34:38.756143 27134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:38.760664 27146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:34:38.766860 27146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:34:38.779552 27146 catalog_manager.cc:1347] Generated new cluster ID: 093a3b0fb5f949fb9a1dca898f28c6f8
I20260806 22:34:38.779748 27146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:34:38.828846 27146 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:34:38.830572 27146 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:34:38.837925 27146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745: Generated new TSK 0
I20260806 22:34:38.839448 27146 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.129.keytab.
Entry for principal oryx/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.129.keytab.
I20260806 22:34:38.941871 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:40953
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.129.keytab
--principal=oryx/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:36635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:39.391142 27160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:39.391602 27160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:39.391793 27160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:39.400148 27160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:39.400362 27160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:39.400528 27160 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:34:39.400676 27160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:34:39.412375 27160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.129.keytab
--principal=oryx/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.198.190:40953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:39.414233 27160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:39.415997 27160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:39.432355 27167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:39.432307 27170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:39.432978 27166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:39.433696 27160 server_base.cc:1034] running on GCE node
I20260806 22:34:39.434489 27160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:39.435717 27160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:39.437000 27160 hybrid_clock.cc:648] HybridClock initialized: now 1786055679436889 us; error 81 us; skew 500 ppm
I20260806 22:34:39.437388 27160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:39 dist-test-slave-h3ft krb5kdc[27000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055679, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:39.441633 27160 init.cc:377] Logged in from keytab as oryx/127.15.198.129@KRBTEST.COM (short username oryx)
I20260806 22:34:39.443497 27160 webserver.cc:460] Webserver started at http://127.15.198.129:33233/ using document root <none> and password file <none>
I20260806 22:34:39.444418 27160 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:39.444516 27160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:39.444849 27160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:39.448954 27160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "7c99e6636e3f4c1fbad3108217e43897"
format_stamp: "Formatted at 2026-08-06 22:34:39 on dist-test-slave-h3ft"
server_key: "035271b9fecb6da7d0a9bc8419952bf0"
server_key_iv: "a735dad0258d76e7b52c596e9c1c0730"
server_key_version: "encryptionkey@0"
I20260806 22:34:39.449883 27160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "7c99e6636e3f4c1fbad3108217e43897"
format_stamp: "Formatted at 2026-08-06 22:34:39 on dist-test-slave-h3ft"
server_key: "035271b9fecb6da7d0a9bc8419952bf0"
server_key_iv: "a735dad0258d76e7b52c596e9c1c0730"
server_key_version: "encryptionkey@0"
I20260806 22:34:39.456411 27160 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 22:34:39.460860 27180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:39.462677 27160 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:34:39.462891 27160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "7c99e6636e3f4c1fbad3108217e43897"
format_stamp: "Formatted at 2026-08-06 22:34:39 on dist-test-slave-h3ft"
server_key: "035271b9fecb6da7d0a9bc8419952bf0"
server_key_iv: "a735dad0258d76e7b52c596e9c1c0730"
server_key_version: "encryptionkey@0"
I20260806 22:34:39.463253 27160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:39.486539 27160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:39.492106 27160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:39.492470 27160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:39.496265 27160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:39.496400 27160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:39.496558 27160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:39.496642 27160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:34:39 dist-test-slave-h3ft krb5kdc[27000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055679, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM
I20260806 22:34:39.528015 27191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:39.497159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:54629 (local address 127.15.198.190:40953)
0806 22:34:39.497846 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:39.497852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:39.499536 (+ 1684us) server_negotiation.cc:407] Connection header received
0806 22:34:39.502283 (+ 2747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:39.502294 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:39.502450 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:39.502609 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:39.506144 (+ 3535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:39.507635 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:39.510307 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:39.510982 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:39.511188 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:39.519177 (+ 7989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:39.519185 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:39.519189 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:39.519241 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:39.523430 (+ 4189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:39.524667 (+ 1237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:39.524673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:39.524676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:39.524820 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:39.525547 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:39.525553 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:39.525557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:39.526066 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:39.526241 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:39.527239 (+ 998us) server_negotiation.cc:299] Negotiation successful
0806 22:34:39.527560 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":394,"threads_started":1}
I20260806 22:34:39.529701 27192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:39.498036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:40953 (local address 127.15.198.129:54629)
0806 22:34:39.499286 (+ 1250us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:39.499372 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:39.501735 (+ 2363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:39.503173 (+ 1438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:39.503193 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:39.504382 (+ 1189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:39.505747 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:39.505778 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:39.507900 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:39.507905 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:39.510105 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:39.510122 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:39.511684 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:39.511697 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:39.512100 (+ 403us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:39.513325 (+ 1225us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:39.513385 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:39.518636 (+ 5251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:39.523674 (+ 5038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:39.523695 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:39.523729 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:39.524489 (+ 760us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:39.525034 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:39.525042 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:39.525048 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:39.525399 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:39.526236 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:39.526246 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:39.526756 (+ 510us) client_negotiation.cc:769] Sending connection context
0806 22:34:39.527124 (+ 368us) client_negotiation.cc:240] Negotiation successful
0806 22:34:39.527897 (+ 773us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":473,"threads_started":1}
I20260806 22:34:39.554219 27160 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:46527
I20260806 22:34:39.554237 27357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:46527 every 8 connection(s)
I20260806 22:34:39.555876 27191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:39.537566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:45313 (local address 127.15.198.190:40953)
0806 22:34:39.537834 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:39.537843 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:39.538612 (+ 769us) server_negotiation.cc:407] Connection header received
0806 22:34:39.538854 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:39.538858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:39.538971 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:39.539131 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:39.540978 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:39.542017 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:39.544572 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:39.545206 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:39.545371 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:39.548638 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:39.548646 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:39.548653 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:39.548707 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:39.552456 (+ 3749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:39.553427 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:39.553436 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:39.553443 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:39.553635 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:39.554062 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:39.554069 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:39.554073 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:39.554508 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:39.554696 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:39.555182 (+ 486us) server_negotiation.cc:299] Negotiation successful
0806 22:34:39.555452 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 22:34:39.555974 27160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:34:39.555982 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:39.537563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:40953 (local address 127.15.198.129:45313)
0806 22:34:39.538458 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:39.538495 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:39.538711 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:39.539171 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:39.539179 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:39.540106 (+ 927us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:39.540677 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:39.540696 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:39.542170 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:39.542176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:39.544371 (+ 2195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:39.544390 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:39.545499 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:39.545509 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:39.545795 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:39.546867 (+ 1072us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:39.546904 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:39.548365 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:39.552624 (+ 4259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:39.552630 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:39.552639 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:39.553261 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:39.553777 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:39.553785 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:39.553788 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:39.553934 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:39.554643 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:39.554648 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:39.554862 (+ 214us) client_negotiation.cc:769] Sending connection context
0806 22:34:39.555065 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 22:34:39.555466 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":467,"threads_started":1}
I20260806 22:34:39.557152 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 27160
I20260806 22:34:39.557466 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:34:39.558848 16154 external_mini_cluster.cc:1428] Setting key 29785b93d4e1478dfa8396ae33bf01da
I20260806 22:34:39.563133 27359 heartbeater.cc:347] Connected to a master server at 127.15.198.190:40953
I20260806 22:34:39.563444 27359 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:39.564406 27359 heartbeater.cc:511] Master 127.15.198.190:40953 requested a full tablet report, sending...
I20260806 22:34:39.567123 27054 ts_manager.cc:194] Registered new tserver with Master: 7c99e6636e3f4c1fbad3108217e43897 (127.15.198.129:46527)
I20260806 22:34:39.569132 27054 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:54629
WARNING: no policy specified for oryx/127.15.198.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.198.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.130.keytab.
Entry for principal oryx/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.130.keytab.
I20260806 22:34:39.635911 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:40953
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.130.keytab
--principal=oryx/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:36635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:40.080935 27364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:40.081418 27364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:40.081609 27364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:40.089880 27364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:40.090122 27364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:40.090297 27364 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:34:40.090448 27364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:34:40.102188 27364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.130.keytab
--principal=oryx/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.198.190:40953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:40.104241 27364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:40.106197 27364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:40.123394 27370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:40.123548 27369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:40.123607 27374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:40.124114 27364 server_base.cc:1034] running on GCE node
I20260806 22:34:40.125000 27364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:40.126283 27364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:40.127528 27364 hybrid_clock.cc:648] HybridClock initialized: now 1786055680127452 us; error 47 us; skew 500 ppm
I20260806 22:34:40.127921 27364 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:40 dist-test-slave-h3ft krb5kdc[27000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055680, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:40.131932 27364 init.cc:377] Logged in from keytab as oryx/127.15.198.130@KRBTEST.COM (short username oryx)
I20260806 22:34:40.133934 27364 webserver.cc:460] Webserver started at http://127.15.198.130:38397/ using document root <none> and password file <none>
I20260806 22:34:40.134864 27364 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:40.135071 27364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:40.135474 27364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:40.139185 27364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "bf501059a592417f8d04320a65d628e1"
format_stamp: "Formatted at 2026-08-06 22:34:40 on dist-test-slave-h3ft"
server_key: "0126b5a9c62b9a9ecadcc58cc0c6ff8c"
server_key_iv: "e87f1472decd831352a8be01db2ffc27"
server_key_version: "encryptionkey@0"
I20260806 22:34:40.140218 27364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "bf501059a592417f8d04320a65d628e1"
format_stamp: "Formatted at 2026-08-06 22:34:40 on dist-test-slave-h3ft"
server_key: "0126b5a9c62b9a9ecadcc58cc0c6ff8c"
server_key_iv: "e87f1472decd831352a8be01db2ffc27"
server_key_version: "encryptionkey@0"
I20260806 22:34:40.146955 27364 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 22:34:40.151572 27385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:40.153401 27364 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260806 22:34:40.153638 27364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "bf501059a592417f8d04320a65d628e1"
format_stamp: "Formatted at 2026-08-06 22:34:40 on dist-test-slave-h3ft"
server_key: "0126b5a9c62b9a9ecadcc58cc0c6ff8c"
server_key_iv: "e87f1472decd831352a8be01db2ffc27"
server_key_version: "encryptionkey@0"
I20260806 22:34:40.153997 27364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:40.192963 27364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:40.199183 27364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:40.199571 27364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:40.203023 27364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:40.203157 27364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 22:34:40.203336 27364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:40.203423 27364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:34:40 dist-test-slave-h3ft krb5kdc[27000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055680, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM
I20260806 22:34:40.239243 27397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:40.203903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:43463 (local address 127.15.198.190:40953)
0806 22:34:40.204606 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:40.204614 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:40.206135 (+ 1521us) server_negotiation.cc:407] Connection header received
0806 22:34:40.209848 (+ 3713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:40.209855 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:40.209975 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:40.210108 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:40.214083 (+ 3975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:40.215300 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:40.219333 (+ 4033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:40.220264 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:40.220438 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:40.227598 (+ 7160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:40.227605 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:40.227609 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:40.227657 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:40.233749 (+ 6092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:40.235363 (+ 1614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:40.235372 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:40.235379 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:40.235661 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:40.236269 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:40.236278 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:40.236284 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:40.236850 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:40.237086 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:40.238108 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0806 22:34:40.238508 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":406,"threads_started":1}
I20260806 22:34:40.240540 27399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:40.204794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:40953 (local address 127.15.198.130:43463)
0806 22:34:40.205869 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:40.205950 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:40.209339 (+ 3389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:40.210564 (+ 1225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:40.210588 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:40.211928 (+ 1340us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:40.213581 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:40.213637 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:40.215544 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:40.215554 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:40.218979 (+ 3425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:40.219041 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:40.221036 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:40.221051 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:40.221644 (+ 593us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:40.222922 (+ 1278us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:40.222962 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:40.227219 (+ 4257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:40.234010 (+ 6791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:40.234022 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:40.234048 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:40.235120 (+ 1072us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:40.235843 (+ 723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:40.235847 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:40.235850 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:40.236114 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:40.237048 (+ 934us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:40.237057 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:40.237582 (+ 525us) client_negotiation.cc:769] Sending connection context
0806 22:34:40.237907 (+ 325us) client_negotiation.cc:240] Negotiation successful
0806 22:34:40.238457 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":424,"threads_started":1}
I20260806 22:34:40.268163 27364 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:39681
I20260806 22:34:40.268185 27558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:39681 every 8 connection(s)
I20260806 22:34:40.269680 27516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:40.251443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:40953 (local address 127.15.198.130:41089)
0806 22:34:40.252266 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:40.252292 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:40.252531 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:40.253049 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:40.253057 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:40.253979 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:40.254531 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:40.254550 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:40.255803 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:40.255814 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:40.257403 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:40.257418 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:40.258418 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:40.258425 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:40.258618 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:40.259686 (+ 1068us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:40.259715 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:40.261179 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:40.266118 (+ 4939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:40.266123 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:40.266127 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:40.266813 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:40.267356 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:40.267362 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:40.267365 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:40.267589 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:40.268518 (+ 929us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:40.268521 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:40.268792 (+ 271us) client_negotiation.cc:769] Sending connection context
0806 22:34:40.268978 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 22:34:40.269298 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":486,"threads_started":1}
I20260806 22:34:40.270164 27364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:34:40.270586 27397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:40.251189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:41089 (local address 127.15.198.190:40953)
0806 22:34:40.251454 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:40.251465 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:40.252432 (+ 967us) server_negotiation.cc:407] Connection header received
0806 22:34:40.252771 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:40.252777 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:40.252865 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:40.252959 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:40.254745 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:40.255638 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:40.257614 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:40.258175 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:40.258312 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:40.261420 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:40.261425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:40.261429 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:40.261469 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:40.265933 (+ 4464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:40.267038 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:40.267044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:40.267048 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:40.267223 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:40.267728 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:40.267734 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:40.267738 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:40.268247 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:40.268436 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:40.270069 (+ 1633us) server_negotiation.cc:299] Negotiation successful
0806 22:34:40.270340 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 22:34:40.272675 27559 heartbeater.cc:347] Connected to a master server at 127.15.198.190:40953
I20260806 22:34:40.274344 27559 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:40.275228 27559 heartbeater.cc:511] Master 127.15.198.190:40953 requested a full tablet report, sending...
I20260806 22:34:40.277073 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 27364
I20260806 22:34:40.277094 27054 ts_manager.cc:194] Registered new tserver with Master: bf501059a592417f8d04320a65d628e1 (127.15.198.130:39681)
I20260806 22:34:40.277333 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:34:40.277944 16154 external_mini_cluster.cc:1428] Setting key 2b0c9f83ec01b0b4e0f6efa6eaecd5a6
I20260806 22:34:40.278122 27054 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:43463
WARNING: no policy specified for oryx/127.15.198.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.198.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.131.keytab.
Entry for principal oryx/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.131.keytab.
I20260806 22:34:40.355940 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:40953
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.131.keytab
--principal=oryx/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:36635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:34:40.573485 27359 heartbeater.cc:503] Master 127.15.198.190:40953 was elected leader, sending a full tablet report...
W20260806 22:34:40.802332 27565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:40.802745 27565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:40.802861 27565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:40.811192 27565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:40.811353 27565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:40.811460 27565 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:34:40.811556 27565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:34:40.822917 27565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.131.keytab
--principal=oryx/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.198.190:40953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:40.824644 27565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:40.826390 27565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:40.843075 27575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:40.843669 27577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:40.843972 27574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:40.848740 27565 server_base.cc:1034] running on GCE node
I20260806 22:34:40.850086 27565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:40.851830 27565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:40.853204 27565 hybrid_clock.cc:648] HybridClock initialized: now 1786055680853096 us; error 77 us; skew 500 ppm
I20260806 22:34:40.853613 27565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:40 dist-test-slave-h3ft krb5kdc[27000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055680, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:40.857831 27565 init.cc:377] Logged in from keytab as oryx/127.15.198.131@KRBTEST.COM (short username oryx)
I20260806 22:34:40.860066 27565 webserver.cc:460] Webserver started at http://127.15.198.131:39135/ using document root <none> and password file <none>
I20260806 22:34:40.861032 27565 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:40.861182 27565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:40.861588 27565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:40.865466 27565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "06011b629c274269b7ec1e8729285b20"
format_stamp: "Formatted at 2026-08-06 22:34:40 on dist-test-slave-h3ft"
server_key: "13e8515d79c97f5fccee69d35d3dc011"
server_key_iv: "620d8e6248b19b6acd1b70e7c09eab42"
server_key_version: "encryptionkey@0"
I20260806 22:34:40.866524 27565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "06011b629c274269b7ec1e8729285b20"
format_stamp: "Formatted at 2026-08-06 22:34:40 on dist-test-slave-h3ft"
server_key: "13e8515d79c97f5fccee69d35d3dc011"
server_key_iv: "620d8e6248b19b6acd1b70e7c09eab42"
server_key_version: "encryptionkey@0"
I20260806 22:34:40.873454 27565 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260806 22:34:40.877871 27586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:40.879798 27565 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:34:40.880028 27565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "06011b629c274269b7ec1e8729285b20"
format_stamp: "Formatted at 2026-08-06 22:34:40 on dist-test-slave-h3ft"
server_key: "13e8515d79c97f5fccee69d35d3dc011"
server_key_iv: "620d8e6248b19b6acd1b70e7c09eab42"
server_key_version: "encryptionkey@0"
I20260806 22:34:40.880327 27565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:40.911306 27565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:40.917415 27565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:40.917825 27565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:40.921582 27565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:40.921730 27565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:40.921921 27565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:40.922026 27565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:34:40 dist-test-slave-h3ft krb5kdc[27000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055680, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM
I20260806 22:34:40.951818 27597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:40.922074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:40771 (local address 127.15.198.190:40953)
0806 22:34:40.922643 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:40.922651 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:40.924168 (+ 1517us) server_negotiation.cc:407] Connection header received
0806 22:34:40.926503 (+ 2335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:40.926510 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:40.926623 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:40.926765 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:40.929625 (+ 2860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:40.931371 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:40.934364 (+ 2993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:40.934877 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:40.935098 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:40.941132 (+ 6034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:40.941144 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:40.941153 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:40.941225 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:40.946396 (+ 5171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:40.947706 (+ 1310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:40.947716 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:40.947724 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:40.947969 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:40.948573 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:40.948582 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:40.948589 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:40.949185 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:40.949403 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:40.950599 (+ 1196us) server_negotiation.cc:299] Negotiation successful
0806 22:34:40.951077 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":279,"threads_started":1}
I20260806 22:34:40.952818 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:40.922825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:40953 (local address 127.15.198.131:40771)
0806 22:34:40.923947 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:40.924001 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:40.926143 (+ 2142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:40.927122 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:40.927144 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:40.928139 (+ 995us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:40.929274 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:40.929299 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:40.931599 (+ 2300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:40.931604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:40.934172 (+ 2568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:40.934190 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:40.935294 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:40.935314 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:40.935577 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:40.936607 (+ 1030us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:40.936635 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:40.940641 (+ 4006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:40.946663 (+ 6022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:40.946682 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:40.946719 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:40.947538 (+ 819us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:40.948125 (+ 587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:40.948129 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:40.948132 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:40.948385 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:40.949396 (+ 1011us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:40.949404 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:40.950065 (+ 661us) client_negotiation.cc:769] Sending connection context
0806 22:34:40.950372 (+ 307us) client_negotiation.cc:240] Negotiation successful
0806 22:34:40.951224 (+ 852us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"thread_start_us":504,"threads_started":1}
I20260806 22:34:40.980873 27597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:40.962391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:37913 (local address 127.15.198.190:40953)
0806 22:34:40.962737 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:40.962749 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:40.963452 (+ 703us) server_negotiation.cc:407] Connection header received
0806 22:34:40.963789 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:40.963799 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:40.963932 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:40.964096 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:40.965313 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:40.967126 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:40.970042 (+ 2916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:40.970790 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:40.970962 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:40.973571 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:40.973581 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:40.973588 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:40.973648 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:40.977218 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:40.978372 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:40.978380 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:40.978385 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:40.978635 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:40.979144 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:40.979153 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:40.979157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:40.979497 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:40.979680 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:40.980170 (+ 490us) server_negotiation.cc:299] Negotiation successful
0806 22:34:40.980437 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260806 22:34:40.980964 27693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:40.962439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:40953 (local address 127.15.198.131:37913)
0806 22:34:40.963298 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:40.963332 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:40.963576 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:40.964155 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:40.964162 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:40.964783 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:40.965130 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:40.965140 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:40.967398 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:40.967407 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:40.969776 (+ 2369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:40.969801 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:40.971074 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:40.971084 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:40.971286 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:40.972365 (+ 1079us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:40.972384 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:40.973326 (+ 942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:40.977428 (+ 4102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:40.977433 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:40.977438 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:40.978223 (+ 785us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:40.978807 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:40.978812 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:40.978816 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:40.978967 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:40.979630 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:40.979634 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:40.979849 (+ 215us) client_negotiation.cc:769] Sending connection context
0806 22:34:40.980023 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 22:34:40.980420 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":457,"threads_started":1}
I20260806 22:34:40.989701 27565 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:41311
I20260806 22:34:40.989737 27749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:41311 every 8 connection(s)
I20260806 22:34:40.991410 27565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:34:40.992045 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 27565
I20260806 22:34:40.992332 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:34:40.993094 16154 external_mini_cluster.cc:1428] Setting key 39c27b7753e35575e6c443f97717ea3b
I20260806 22:34:40.994824 27751 heartbeater.cc:347] Connected to a master server at 127.15.198.190:40953
I20260806 22:34:40.995126 27751 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:40.996016 27751 heartbeater.cc:511] Master 127.15.198.190:40953 requested a full tablet report, sending...
I20260806 22:34:40.997689 27054 ts_manager.cc:194] Registered new tserver with Master: 06011b629c274269b7ec1e8729285b20 (127.15.198.131:41311)
I20260806 22:34:40.998800 27054 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:40771
I20260806 22:34:41.010038 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:34:41 dist-test-slave-h3ft krb5kdc[27000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055681, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:34:41 dist-test-slave-h3ft krb5kdc[27000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:41 dist-test-slave-h3ft krb5kdc[27000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:34:41.088289 27597 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:41.075063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60442 (local address 127.15.198.190:40953)
0806 22:34:41.075343 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:41.075352 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:41.075526 (+ 174us) server_negotiation.cc:407] Connection header received
0806 22:34:41.075714 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:41.075719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:41.075823 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:41.075958 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:41.077653 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.078565 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.080121 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.080598 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.080715 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.087919 (+ 7204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60442: BlockingRecv error: recv got EOF from 127.0.0.1:60442 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 06 22:34:41 dist-test-slave-h3ft krb5kdc[27000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055681, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM
I20260806 22:34:41.109239 27597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.092214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60456 (local address 127.15.198.190:40953)
0806 22:34:41.092466 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:41.092471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:41.092844 (+ 373us) server_negotiation.cc:407] Connection header received
0806 22:34:41.093029 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:41.093034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:41.093117 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:41.093225 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:41.094702 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.095671 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.097112 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.097617 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.097739 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.102661 (+ 4922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:41.102668 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:41.102672 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:41.102731 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:41.106093 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:41.107101 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:41.107106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:41.107109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:41.107275 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:41.107699 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:41.107703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:41.107705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:41.108077 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:41.108329 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:41.108685 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 22:34:41.108942 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 22:34:41.114588 27054 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:34:41.122741 27054 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:34:41.175165 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.141258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:39681 (local address 127.0.0.1:53512)
0806 22:34:41.162077 (+ 20819us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:41.162108 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:41.162348 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:41.163851 (+ 1503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:41.163864 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:41.163932 (+ 68us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:41.166044 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.166064 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.168528 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.168533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.171444 (+ 2911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.171462 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.173969 (+ 2507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.173977 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.174214 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.174282 (+ 68us) client_negotiation.cc:769] Sending connection context
0806 22:34:41.174450 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 22:34:41.174691 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20579,"thread_start_us":21242,"threads_started":1}
I20260806 22:34:41.175184 27780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.141455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53512 (local address 127.15.198.130:39681)
0806 22:34:41.153720 (+ 12265us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:41.153735 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:41.162281 (+ 8546us) server_negotiation.cc:407] Connection header received
0806 22:34:41.162609 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:41.162625 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:41.163141 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:41.163454 (+ 313us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:41.166325 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.168308 (+ 1983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.171700 (+ 3392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.173353 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.173552 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.173772 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:41.174546 (+ 774us) server_negotiation.cc:299] Negotiation successful
0806 22:34:41.174777 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12016,"thread_start_us":11809,"threads_started":1}
I20260806 22:34:41.176165 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.140899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:46527 (local address 127.0.0.1:41304)
0806 22:34:41.154470 (+ 13571us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:41.154546 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:41.154788 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:41.166529 (+ 11741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:41.166534 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:41.166562 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:41.167019 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.167036 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.170761 (+ 3725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.170770 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.173333 (+ 2563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.173348 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.175108 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.175115 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.175526 (+ 411us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.175560 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 22:34:41.175629 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:34:41.175770 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13159,"thread_start_us":12923,"threads_started":1}
I20260806 22:34:41.177311 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.141682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41304 (local address 127.15.198.129:46527)
0806 22:34:41.154969 (+ 13287us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:41.154980 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:41.165151 (+ 10171us) server_negotiation.cc:407] Connection header received
0806 22:34:41.165303 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:41.165311 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:41.165591 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:41.167302 (+ 1711us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:41.167567 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.170554 (+ 2987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.173558 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.174832 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.176494 (+ 1662us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.176688 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:41.176796 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 22:34:41.176990 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13069,"thread_start_us":13304,"threads_started":1}
I20260806 22:34:41.179926 27783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.155651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:41311 (local address 127.0.0.1:56678)
0806 22:34:41.171346 (+ 15695us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:41.171373 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:41.171572 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:41.173099 (+ 1527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:41.173103 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:41.173124 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:41.173558 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.173569 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.175733 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.175739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.177630 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.177645 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.179147 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.179154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.179327 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.179359 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 22:34:41.179444 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:34:41.179591 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15512,"thread_start_us":15265,"threads_started":1}
I20260806 22:34:41.180308 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.155625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56678 (local address 127.15.198.131:41311)
0806 22:34:41.168637 (+ 13012us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:41.168650 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:41.171828 (+ 3178us) server_negotiation.cc:407] Connection header received
0806 22:34:41.171990 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:41.172001 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:41.172353 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:41.172616 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:41.173825 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.175569 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.177837 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.178894 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.179203 (+ 309us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.179416 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:41.179494 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 22:34:41.179673 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12750,"thread_start_us":13061,"threads_started":1}
I20260806 22:34:41.183224 27251 tablet_service.cc:1467] Processing CreateTablet for tablet 6dddd480e76e49f6a8cb377336fb388a (DEFAULT_TABLE table=test-table [id=fabb67328aeb46ed82267cffcbe47d79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:41.183578 27655 tablet_service.cc:1467] Processing CreateTablet for tablet 6dddd480e76e49f6a8cb377336fb388a (DEFAULT_TABLE table=test-table [id=fabb67328aeb46ed82267cffcbe47d79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:41.184883 27251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dddd480e76e49f6a8cb377336fb388a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:41.185011 27655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dddd480e76e49f6a8cb377336fb388a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:41.189471 27461 tablet_service.cc:1467] Processing CreateTablet for tablet 6dddd480e76e49f6a8cb377336fb388a (DEFAULT_TABLE table=test-table [id=fabb67328aeb46ed82267cffcbe47d79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:41.191498 27461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dddd480e76e49f6a8cb377336fb388a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:41.199368 27784 tablet_bootstrap.cc:492] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20: Bootstrap starting.
I20260806 22:34:41.203608 27784 tablet_bootstrap.cc:654] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:41.204062 27786 tablet_bootstrap.cc:492] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897: Bootstrap starting.
I20260806 22:34:41.205547 27784 log.cc:824] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:41.209194 27786 tablet_bootstrap.cc:654] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:41.209304 27789 tablet_bootstrap.cc:492] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1: Bootstrap starting.
I20260806 22:34:41.210939 27786 log.cc:824] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:41.213436 27789 tablet_bootstrap.cc:654] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:41.213886 27784 tablet_bootstrap.cc:492] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20: No bootstrap required, opened a new log
I20260806 22:34:41.214265 27784 ts_tablet_manager.cc:1397] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.003s
I20260806 22:34:41.214510 27786 tablet_bootstrap.cc:492] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897: No bootstrap required, opened a new log
I20260806 22:34:41.214807 27789 log.cc:824] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:41.214821 27786 ts_tablet_manager.cc:1397] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 22:34:41.217590 27789 tablet_bootstrap.cc:492] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1: No bootstrap required, opened a new log
I20260806 22:34:41.217891 27789 ts_tablet_manager.cc:1397] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260806 22:34:41.226639 27784 raft_consensus.cc:348] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.226624 27789 raft_consensus.cc:348] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.226637 27786 raft_consensus.cc:348] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.227214 27789 raft_consensus.cc:374] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:41.227218 27786 raft_consensus.cc:374] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:41.227218 27784 raft_consensus.cc:374] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:41.227341 27786 raft_consensus.cc:729] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c99e6636e3f4c1fbad3108217e43897, State: Initialized, Role: FOLLOWER
I20260806 22:34:41.227339 27789 raft_consensus.cc:729] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf501059a592417f8d04320a65d628e1, State: Initialized, Role: FOLLOWER
I20260806 22:34:41.227336 27784 raft_consensus.cc:729] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06011b629c274269b7ec1e8729285b20, State: Initialized, Role: FOLLOWER
I20260806 22:34:41.228157 27789 consensus_queue.cc:260] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.228169 27786 consensus_queue.cc:260] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.228168 27784 consensus_queue.cc:260] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.230425 27751 heartbeater.cc:503] Master 127.15.198.190:40953 was elected leader, sending a full tablet report...
I20260806 22:34:41.231120 27784 ts_tablet_manager.cc:1428] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20: Time spent starting tablet: real 0.017s user 0.018s sys 0.002s
I20260806 22:34:41.231868 27786 ts_tablet_manager.cc:1428] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897: Time spent starting tablet: real 0.017s user 0.018s sys 0.000s
I20260806 22:34:41.233443 27559 heartbeater.cc:503] Master 127.15.198.190:40953 was elected leader, sending a full tablet report...
I20260806 22:34:41.234222 27789 ts_tablet_manager.cc:1428] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
W20260806 22:34:41.243767 27757 tablet.cc:2369] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:34:41.265050 27795 raft_consensus.cc:482] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:34:41.265596 27795 raft_consensus.cc:504] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.268743 27795 leader_election.cc:288] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c99e6636e3f4c1fbad3108217e43897 (127.15.198.129:46527), 06011b629c274269b7ec1e8729285b20 (127.15.198.131:41311)
W20260806 22:34:41.272898 27560 tablet.cc:2369] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:34:41.279084 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.269340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:48195 (local address 127.15.198.129:46527)
0806 22:34:41.269677 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:41.269685 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:41.270137 (+ 452us) server_negotiation.cc:407] Connection header received
0806 22:34:41.270569 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:41.270576 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:41.270682 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:41.270755 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:41.271769 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.272982 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.276317 (+ 3335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.277693 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.277879 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.277969 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:41.278562 (+ 593us) server_negotiation.cc:299] Negotiation successful
0806 22:34:41.278756 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260806 22:34:41.279376 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.270470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:39839 (local address 127.15.198.131:41311)
0806 22:34:41.270719 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:41.270729 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:41.271345 (+ 616us) server_negotiation.cc:407] Connection header received
0806 22:34:41.271600 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:41.271605 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:41.271713 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:41.271823 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:41.272534 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.273807 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.276638 (+ 2831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.277633 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.277849 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.277984 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:41.278705 (+ 721us) server_negotiation.cc:299] Negotiation successful
0806 22:34:41.278946 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:34:41.280390 27799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.270104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:41311 (local address 127.15.198.130:39839)
0806 22:34:41.271195 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:41.271217 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:41.271404 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:41.271843 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:41.271848 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:41.271872 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:41.272339 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.272352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.274005 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.274014 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.276461 (+ 2447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.276472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.278219 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.278230 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.278463 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.278502 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 22:34:41.279574 (+ 1072us) client_negotiation.cc:240] Negotiation successful
0806 22:34:41.279778 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":930,"mutex_wait_us":1,"thread_start_us":631,"threads_started":1}
I20260806 22:34:41.280416 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.269005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:46527 (local address 127.15.198.130:48195)
0806 22:34:41.270000 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:41.270030 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:41.270333 (+ 303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:41.271049 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:41.271054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:41.271085 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:41.271594 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.271607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.273189 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.273195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.276122 (+ 2927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.276140 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.278150 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.278158 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.278357 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.278396 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 22:34:41.279700 (+ 1304us) client_negotiation.cc:240] Negotiation successful
0806 22:34:41.279834 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"mutex_wait_us":28,"thread_start_us":468,"threads_started":1}
I20260806 22:34:41.281742 27687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dddd480e76e49f6a8cb377336fb388a" candidate_uuid: "bf501059a592417f8d04320a65d628e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06011b629c274269b7ec1e8729285b20" is_pre_election: true
I20260806 22:34:41.281750 27283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dddd480e76e49f6a8cb377336fb388a" candidate_uuid: "bf501059a592417f8d04320a65d628e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c99e6636e3f4c1fbad3108217e43897" is_pre_election: true
I20260806 22:34:41.282459 27687 raft_consensus.cc:2454] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf501059a592417f8d04320a65d628e1 in term 0.
I20260806 22:34:41.282461 27283 raft_consensus.cc:2454] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf501059a592417f8d04320a65d628e1 in term 0.
I20260806 22:34:41.283285 27390 leader_election.cc:302] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06011b629c274269b7ec1e8729285b20, bf501059a592417f8d04320a65d628e1; no voters:
I20260806 22:34:41.283805 27795 raft_consensus.cc:2781] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:34:41.283984 27795 raft_consensus.cc:482] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:34:41.284101 27795 raft_consensus.cc:3037] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:41.286964 27795 raft_consensus.cc:504] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.287662 27795 leader_election.cc:288] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [CANDIDATE]: Term 1 election: Requested vote from peers 7c99e6636e3f4c1fbad3108217e43897 (127.15.198.129:46527), 06011b629c274269b7ec1e8729285b20 (127.15.198.131:41311)
I20260806 22:34:41.288370 27283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dddd480e76e49f6a8cb377336fb388a" candidate_uuid: "bf501059a592417f8d04320a65d628e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c99e6636e3f4c1fbad3108217e43897"
I20260806 22:34:41.288472 27687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dddd480e76e49f6a8cb377336fb388a" candidate_uuid: "bf501059a592417f8d04320a65d628e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06011b629c274269b7ec1e8729285b20"
I20260806 22:34:41.288618 27283 raft_consensus.cc:3037] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:41.288831 27687 raft_consensus.cc:3037] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:41.291709 27283 raft_consensus.cc:2454] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf501059a592417f8d04320a65d628e1 in term 1.
I20260806 22:34:41.291994 27687 raft_consensus.cc:2454] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf501059a592417f8d04320a65d628e1 in term 1.
I20260806 22:34:41.292318 27390 leader_election.cc:302] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c99e6636e3f4c1fbad3108217e43897, bf501059a592417f8d04320a65d628e1; no voters:
I20260806 22:34:41.292718 27795 raft_consensus.cc:2781] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:41.293289 27795 raft_consensus.cc:686] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [term 1 LEADER]: Becoming Leader. State: Replica: bf501059a592417f8d04320a65d628e1, State: Running, Role: LEADER
I20260806 22:34:41.294018 27795 consensus_queue.cc:237] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.300712 27053 catalog_manager.cc:5495] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf501059a592417f8d04320a65d628e1 (127.15.198.130). New cstate: current_term: 1 leader_uuid: "bf501059a592417f8d04320a65d628e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } health_report { overall_health: UNKNOWN } } }
W20260806 22:34:41.317790 27360 tablet.cc:2369] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:34:41.354102 27780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.347721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53520 (local address 127.15.198.130:39681)
0806 22:34:41.347935 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:41.347940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:41.347961 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:34:41.348184 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:41.348188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:41.348298 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:41.348446 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:34:41.348836 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.349645 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.351382 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.351833 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.351944 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.352504 (+ 560us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:34:41.353273 (+ 769us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:34:41.353416 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:41.353605 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 22:34:41.353744 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 22:34:41.365444 27687 raft_consensus.cc:1261] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Refusing update from remote peer bf501059a592417f8d04320a65d628e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:41.365445 27283 raft_consensus.cc:1261] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897 [term 1 FOLLOWER]: Refusing update from remote peer bf501059a592417f8d04320a65d628e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:41.366740 27800 consensus_queue.cc:1035] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:41.367278 27795 consensus_queue.cc:1035] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:41.383114 27803 mvcc.cc:205] Tried to move back new op lower bound from 7315684070858911744 to 7315684070585356288. Current Snapshot: MvccSnapshot[applied={T|T < 7315684070858911744}]
I20260806 22:34:41.385314 27807 mvcc.cc:205] Tried to move back new op lower bound from 7315684070858911744 to 7315684070585356288. Current Snapshot: MvccSnapshot[applied={T|T < 7315684070858911744}]
I20260806 22:34:41.402244 27053 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:34:41.402671 27053 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:34:41.407887 27053 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745: Sending DeleteTablet for 3 replicas of tablet 6dddd480e76e49f6a8cb377336fb388a
I20260806 22:34:41.409720 27461 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dddd480e76e49f6a8cb377336fb388a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:41 UTC) from {username='oryx', principal='oryx/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:53512
I20260806 22:34:41.409799 27251 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dddd480e76e49f6a8cb377336fb388a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:41 UTC) from {username='oryx', principal='oryx/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:41304
I20260806 22:34:41.409811 27655 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dddd480e76e49f6a8cb377336fb388a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:41 UTC) from {username='oryx', principal='oryx/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:56678
I20260806 22:34:41.412262 27053 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:34:41.413618 27053 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:34:41.422478 27818 tablet_replica.cc:330] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897: stopping tablet replica
I20260806 22:34:41.423280 27818 raft_consensus.cc:2229] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:34:41.424055 27818 raft_consensus.cc:2258] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:41.424096 27819 tablet_replica.cc:330] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20: stopping tablet replica
I20260806 22:34:41.425134 27816 tablet_replica.cc:330] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1: stopping tablet replica
I20260806 22:34:41.425673 27816 raft_consensus.cc:2229] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:34:41.426604 27816 raft_consensus.cc:2258] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:41.427896 27819 raft_consensus.cc:2229] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:34:41.428610 27819 raft_consensus.cc:2258] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:41.429524 27818 ts_tablet_manager.cc:1905] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:41.431551 27461 tablet_service.cc:1467] Processing CreateTablet for tablet 8ac44d59ebfa444f8d0e3e7cab6f6984 (DEFAULT_TABLE table=test-table [id=7003bd83e1004d90a23c02f8d8d6436c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:41.431586 27655 tablet_service.cc:1467] Processing CreateTablet for tablet 8ac44d59ebfa444f8d0e3e7cab6f6984 (DEFAULT_TABLE table=test-table [id=7003bd83e1004d90a23c02f8d8d6436c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:41.432183 27655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ac44d59ebfa444f8d0e3e7cab6f6984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:41.432219 27461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ac44d59ebfa444f8d0e3e7cab6f6984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:41.432953 27816 ts_tablet_manager.cc:1905] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:41.440883 27251 tablet_service.cc:1467] Processing CreateTablet for tablet 8ac44d59ebfa444f8d0e3e7cab6f6984 (DEFAULT_TABLE table=test-table [id=7003bd83e1004d90a23c02f8d8d6436c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:41.441161 27816 ts_tablet_manager.cc:1918] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:34:41.441401 27816 log.cc:1196] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/6dddd480e76e49f6a8cb377336fb388a
I20260806 22:34:41.441551 27251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ac44d59ebfa444f8d0e3e7cab6f6984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:41.442065 27816 ts_tablet_manager.cc:1939] T 6dddd480e76e49f6a8cb377336fb388a P bf501059a592417f8d04320a65d628e1: Deleting consensus metadata
I20260806 22:34:41.442219 27819 ts_tablet_manager.cc:1905] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:41.444621 27036 catalog_manager.cc:4841] TS bf501059a592417f8d04320a65d628e1 (127.15.198.130:39681): tablet 6dddd480e76e49f6a8cb377336fb388a (table test-table [id=fabb67328aeb46ed82267cffcbe47d79]) successfully deleted
I20260806 22:34:41.446300 27789 tablet_bootstrap.cc:492] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1: Bootstrap starting.
I20260806 22:34:41.446898 27784 tablet_bootstrap.cc:492] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20: Bootstrap starting.
I20260806 22:34:41.448323 27789 tablet_bootstrap.cc:654] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:41.449045 27784 tablet_bootstrap.cc:654] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:41.455510 27819 ts_tablet_manager.cc:1918] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:34:41.455889 27819 log.cc:1196] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/6dddd480e76e49f6a8cb377336fb388a
I20260806 22:34:41.456525 27819 ts_tablet_manager.cc:1939] T 6dddd480e76e49f6a8cb377336fb388a P 06011b629c274269b7ec1e8729285b20: Deleting consensus metadata
I20260806 22:34:41.459787 27040 catalog_manager.cc:4841] TS 06011b629c274269b7ec1e8729285b20 (127.15.198.131:41311): tablet 6dddd480e76e49f6a8cb377336fb388a (table test-table [id=fabb67328aeb46ed82267cffcbe47d79]) successfully deleted
I20260806 22:34:41.459869 27818 ts_tablet_manager.cc:1918] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:34:41.460207 27818 log.cc:1196] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/6dddd480e76e49f6a8cb377336fb388a
I20260806 22:34:41.461206 27818 ts_tablet_manager.cc:1939] T 6dddd480e76e49f6a8cb377336fb388a P 7c99e6636e3f4c1fbad3108217e43897: Deleting consensus metadata
I20260806 22:34:41.461416 27784 tablet_bootstrap.cc:492] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20: No bootstrap required, opened a new log
I20260806 22:34:41.461628 27784 ts_tablet_manager.cc:1397] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.002s
I20260806 22:34:41.462967 27784 raft_consensus.cc:348] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.464617 27040 catalog_manager.cc:4841] TS 7c99e6636e3f4c1fbad3108217e43897 (127.15.198.129:46527): tablet 6dddd480e76e49f6a8cb377336fb388a (table test-table [id=fabb67328aeb46ed82267cffcbe47d79]) successfully deleted
I20260806 22:34:41.463346 27784 raft_consensus.cc:374] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:41.465055 27784 raft_consensus.cc:729] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06011b629c274269b7ec1e8729285b20, State: Initialized, Role: FOLLOWER
I20260806 22:34:41.465620 27784 consensus_queue.cc:260] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.466923 27789 tablet_bootstrap.cc:492] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1: No bootstrap required, opened a new log
I20260806 22:34:41.467240 27789 ts_tablet_manager.cc:1397] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1: Time spent bootstrapping tablet: real 0.021s user 0.011s sys 0.004s
I20260806 22:34:41.467425 27786 tablet_bootstrap.cc:492] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897: Bootstrap starting.
I20260806 22:34:41.468477 27789 raft_consensus.cc:348] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.468868 27789 raft_consensus.cc:374] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:41.469033 27789 raft_consensus.cc:729] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf501059a592417f8d04320a65d628e1, State: Initialized, Role: FOLLOWER
I20260806 22:34:41.469583 27786 tablet_bootstrap.cc:654] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:41.469883 27784 ts_tablet_manager.cc:1428] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20: Time spent starting tablet: real 0.008s user 0.000s sys 0.004s
I20260806 22:34:41.469483 27789 consensus_queue.cc:260] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.472530 27789 ts_tablet_manager.cc:1428] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260806 22:34:41.478505 27786 tablet_bootstrap.cc:492] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897: No bootstrap required, opened a new log
I20260806 22:34:41.478787 27786 ts_tablet_manager.cc:1397] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.009s
I20260806 22:34:41.479740 27811 raft_consensus.cc:482] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:34:41.479899 27786 raft_consensus.cc:348] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.480216 27786 raft_consensus.cc:374] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:41.480351 27786 raft_consensus.cc:729] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c99e6636e3f4c1fbad3108217e43897, State: Initialized, Role: FOLLOWER
I20260806 22:34:41.480067 27811 raft_consensus.cc:504] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.480831 27786 consensus_queue.cc:260] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.481293 27811 leader_election.cc:288] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c99e6636e3f4c1fbad3108217e43897 (127.15.198.129:46527), 06011b629c274269b7ec1e8729285b20 (127.15.198.131:41311)
I20260806 22:34:41.482040 27687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ac44d59ebfa444f8d0e3e7cab6f6984" candidate_uuid: "bf501059a592417f8d04320a65d628e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06011b629c274269b7ec1e8729285b20" is_pre_election: true
I20260806 22:34:41.482411 27687 raft_consensus.cc:2454] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf501059a592417f8d04320a65d628e1 in term 0.
I20260806 22:34:41.482399 27283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ac44d59ebfa444f8d0e3e7cab6f6984" candidate_uuid: "bf501059a592417f8d04320a65d628e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c99e6636e3f4c1fbad3108217e43897" is_pre_election: true
I20260806 22:34:41.482664 27283 raft_consensus.cc:2454] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf501059a592417f8d04320a65d628e1 in term 0.
I20260806 22:34:41.483387 27786 ts_tablet_manager.cc:1428] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260806 22:34:41.483362 27390 leader_election.cc:302] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06011b629c274269b7ec1e8729285b20, bf501059a592417f8d04320a65d628e1; no voters:
I20260806 22:34:41.483775 27811 raft_consensus.cc:2781] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:34:41.483978 27811 raft_consensus.cc:482] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:34:41.484109 27811 raft_consensus.cc:3037] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:41.488364 27811 raft_consensus.cc:504] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.489836 27811 leader_election.cc:288] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [CANDIDATE]: Term 1 election: Requested vote from peers 7c99e6636e3f4c1fbad3108217e43897 (127.15.198.129:46527), 06011b629c274269b7ec1e8729285b20 (127.15.198.131:41311)
I20260806 22:34:41.490360 27283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ac44d59ebfa444f8d0e3e7cab6f6984" candidate_uuid: "bf501059a592417f8d04320a65d628e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c99e6636e3f4c1fbad3108217e43897"
I20260806 22:34:41.490444 27687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ac44d59ebfa444f8d0e3e7cab6f6984" candidate_uuid: "bf501059a592417f8d04320a65d628e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06011b629c274269b7ec1e8729285b20"
I20260806 22:34:41.490636 27283 raft_consensus.cc:3037] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:41.490763 27687 raft_consensus.cc:3037] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:41.493346 27687 raft_consensus.cc:2454] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf501059a592417f8d04320a65d628e1 in term 1.
I20260806 22:34:41.494115 27283 raft_consensus.cc:2454] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf501059a592417f8d04320a65d628e1 in term 1.
I20260806 22:34:41.494211 27390 leader_election.cc:302] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06011b629c274269b7ec1e8729285b20, bf501059a592417f8d04320a65d628e1; no voters:
I20260806 22:34:41.495235 27811 raft_consensus.cc:2781] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:41.495469 27811 raft_consensus.cc:686] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [term 1 LEADER]: Becoming Leader. State: Replica: bf501059a592417f8d04320a65d628e1, State: Running, Role: LEADER
I20260806 22:34:41.495882 27811 consensus_queue.cc:237] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.499576 27053 catalog_manager.cc:5495] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf501059a592417f8d04320a65d628e1 (127.15.198.130). New cstate: current_term: 1 leader_uuid: "bf501059a592417f8d04320a65d628e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:34:41 dist-test-slave-h3ft krb5kdc[27000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055678, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.190@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM
I20260806 22:34:41.558605 27833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.538539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:40953 (local address 127.0.0.1:60468)
0806 22:34:41.539529 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:41.539560 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:41.539735 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:41.540246 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:41.540253 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:41.541629 (+ 1376us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:41.542021 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.542037 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.543318 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.543324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.544979 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.544996 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.545870 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.545879 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.546070 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.547024 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:41.547055 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:41.550653 (+ 3598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:41.555046 (+ 4393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:41.555063 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:41.555073 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:41.555736 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:41.556235 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:41.556242 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:41.556248 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:41.556482 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:41.557236 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:41.557247 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:41.557491 (+ 244us) client_negotiation.cc:769] Sending connection context
0806 22:34:41.557667 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 22:34:41.558023 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":594,"threads_started":1}
I20260806 22:34:41.559295 27597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.538539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60468 (local address 127.15.198.190:40953)
0806 22:34:41.538869 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:41.538880 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:41.539660 (+ 780us) server_negotiation.cc:407] Connection header received
0806 22:34:41.539886 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:41.539890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:41.539992 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:41.540103 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:41.542219 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.543179 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.545131 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.545589 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.545778 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.550919 (+ 5141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:41.550926 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:41.550932 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:41.550975 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:41.554829 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:41.555871 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:41.555878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:41.555883 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:41.556048 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:41.556644 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:41.556652 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:41.556658 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:41.557075 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:41.557244 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:41.557808 (+ 564us) server_negotiation.cc:299] Negotiation successful
0806 22:34:41.558227 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":126,"server-negotiator.queue_time_us":145}
I20260806 22:34:41.566488 27053 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:60468:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:34:41.568174 27053 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:34:41.577629 27251 tablet_service.cc:1467] Processing CreateTablet for tablet 039d3b3a379f466a9dcbef39becacf74 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=970a6b2e10cd48508dff7a8c8a094bb4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:34:41.578627 27251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 039d3b3a379f466a9dcbef39becacf74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:41.578558 27655 tablet_service.cc:1467] Processing CreateTablet for tablet 039d3b3a379f466a9dcbef39becacf74 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=970a6b2e10cd48508dff7a8c8a094bb4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:34:41.578964 27461 tablet_service.cc:1467] Processing CreateTablet for tablet 039d3b3a379f466a9dcbef39becacf74 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=970a6b2e10cd48508dff7a8c8a094bb4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:34:41.579509 27655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 039d3b3a379f466a9dcbef39becacf74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:41.579898 27461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 039d3b3a379f466a9dcbef39becacf74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:41.590040 27786 tablet_bootstrap.cc:492] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897: Bootstrap starting.
I20260806 22:34:41.592877 27786 tablet_bootstrap.cc:654] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:41.593811 27784 tablet_bootstrap.cc:492] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20: Bootstrap starting.
I20260806 22:34:41.596935 27784 tablet_bootstrap.cc:654] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:41.598119 27789 tablet_bootstrap.cc:492] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1: Bootstrap starting.
I20260806 22:34:41.600488 27789 tablet_bootstrap.cc:654] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:41.602326 27786 tablet_bootstrap.cc:492] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897: No bootstrap required, opened a new log
I20260806 22:34:41.602726 27786 ts_tablet_manager.cc:1397] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260806 22:34:41.604686 27786 raft_consensus.cc:348] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.605270 27786 raft_consensus.cc:374] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:41.605396 27786 raft_consensus.cc:729] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c99e6636e3f4c1fbad3108217e43897, State: Initialized, Role: FOLLOWER
I20260806 22:34:41.605845 27786 consensus_queue.cc:260] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.606570 27796 tablet_replica.cc:439] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } } }
I20260806 22:34:41.607163 27796 tablet_replica.cc:442] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:34:41.607343 27784 tablet_bootstrap.cc:492] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20: No bootstrap required, opened a new log
I20260806 22:34:41.607583 27784 ts_tablet_manager.cc:1397] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260806 22:34:41.608837 27786 ts_tablet_manager.cc:1428] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260806 22:34:41.608848 27784 raft_consensus.cc:348] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.609196 27784 raft_consensus.cc:374] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:41.609280 27784 raft_consensus.cc:729] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06011b629c274269b7ec1e8729285b20, State: Initialized, Role: FOLLOWER
I20260806 22:34:41.609625 27784 consensus_queue.cc:260] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.610635 27784 ts_tablet_manager.cc:1428] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260806 22:34:41.610457 27797 tablet_replica.cc:439] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } } }
I20260806 22:34:41.610987 27797 tablet_replica.cc:442] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:34:41.618882 27789 tablet_bootstrap.cc:492] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1: No bootstrap required, opened a new log
I20260806 22:34:41.619215 27789 ts_tablet_manager.cc:1397] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.004s
I20260806 22:34:41.620255 27796 raft_consensus.cc:482] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:34:41.620288 27789 raft_consensus.cc:348] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.620622 27789 raft_consensus.cc:374] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:41.620719 27789 raft_consensus.cc:729] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf501059a592417f8d04320a65d628e1, State: Initialized, Role: FOLLOWER
I20260806 22:34:41.620520 27796 raft_consensus.cc:504] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.621035 27789 consensus_queue.cc:260] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.621639 27811 tablet_replica.cc:439] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } } }
I20260806 22:34:41.621994 27811 tablet_replica.cc:442] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:34:41.623253 27796 leader_election.cc:288] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf501059a592417f8d04320a65d628e1 (127.15.198.130:39681), 06011b629c274269b7ec1e8729285b20 (127.15.198.131:41311)
I20260806 22:34:41.635267 27780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.627215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:45697 (local address 127.15.198.130:39681)
0806 22:34:41.627467 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:41.627475 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:41.627497 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:34:41.627588 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:41.627596 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:41.627687 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:41.627804 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:41.628513 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.629491 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.632001 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.633531 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.633772 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.633882 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:41.634608 (+ 726us) server_negotiation.cc:299] Negotiation successful
0806 22:34:41.634801 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 22:34:41.635525 27837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.623337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:39681 (local address 127.15.198.129:45697)
0806 22:34:41.626493 (+ 3156us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:41.626528 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:41.626717 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:41.627846 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:41.627852 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:41.627889 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:41.628321 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.628335 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.629642 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.629651 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.631696 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.631714 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.634135 (+ 2421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.634144 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.634367 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.634407 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 22:34:41.634881 (+ 474us) client_negotiation.cc:240] Negotiation successful
0806 22:34:41.635128 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2952,"thread_start_us":4275,"threads_started":1}
I20260806 22:34:41.636626 27491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039d3b3a379f466a9dcbef39becacf74" candidate_uuid: "7c99e6636e3f4c1fbad3108217e43897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf501059a592417f8d04320a65d628e1" is_pre_election: true
I20260806 22:34:41.637336 27491 raft_consensus.cc:2454] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c99e6636e3f4c1fbad3108217e43897 in term 0.
I20260806 22:34:41.638245 27181 leader_election.cc:302] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c99e6636e3f4c1fbad3108217e43897, bf501059a592417f8d04320a65d628e1; no voters:
I20260806 22:34:41.634835 27789 ts_tablet_manager.cc:1428] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 22:34:41.638799 27796 raft_consensus.cc:2781] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:34:41.638950 27796 raft_consensus.cc:482] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:34:41.639099 27796 raft_consensus.cc:3037] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:41.642799 27796 raft_consensus.cc:504] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.643949 27796 leader_election.cc:288] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [CANDIDATE]: Term 1 election: Requested vote from peers bf501059a592417f8d04320a65d628e1 (127.15.198.130:39681), 06011b629c274269b7ec1e8729285b20 (127.15.198.131:41311)
I20260806 22:34:41.644819 27491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039d3b3a379f466a9dcbef39becacf74" candidate_uuid: "7c99e6636e3f4c1fbad3108217e43897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf501059a592417f8d04320a65d628e1"
I20260806 22:34:41.645131 27491 raft_consensus.cc:3037] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:41.648275 27837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.626462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:41311 (local address 127.15.198.129:46661)
0806 22:34:41.636407 (+ 9945us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:41.636436 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:41.636720 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:41.637406 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:41.637411 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:41.637445 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:41.637663 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.637681 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.642199 (+ 4518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.642205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.644865 (+ 2660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.644886 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.647064 (+ 2178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.647109 (+ 45us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.647375 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.647439 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 22:34:41.647633 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 22:34:41.647851 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9802,"thread_start_us":13236,"threads_started":1}
I20260806 22:34:41.648319 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.627398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:46661 (local address 127.15.198.131:41311)
0806 22:34:41.627611 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:41.627617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:41.636552 (+ 8935us) server_negotiation.cc:407] Connection header received
0806 22:34:41.636950 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:41.636958 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:41.637083 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:41.637157 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:41.640563 (+ 3406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.641893 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.645108 (+ 3215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.646588 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.646757 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.646852 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:41.647741 (+ 889us) server_negotiation.cc:299] Negotiation successful
0806 22:34:41.647949 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 22:34:41.649335 27687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039d3b3a379f466a9dcbef39becacf74" candidate_uuid: "7c99e6636e3f4c1fbad3108217e43897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06011b629c274269b7ec1e8729285b20" is_pre_election: true
I20260806 22:34:41.649575 27686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039d3b3a379f466a9dcbef39becacf74" candidate_uuid: "7c99e6636e3f4c1fbad3108217e43897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06011b629c274269b7ec1e8729285b20"
I20260806 22:34:41.649762 27687 raft_consensus.cc:2454] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c99e6636e3f4c1fbad3108217e43897 in term 0.
I20260806 22:34:41.649940 27686 raft_consensus.cc:3037] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:41.651216 27491 raft_consensus.cc:2454] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c99e6636e3f4c1fbad3108217e43897 in term 1.
I20260806 22:34:41.651932 27181 leader_election.cc:302] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c99e6636e3f4c1fbad3108217e43897, bf501059a592417f8d04320a65d628e1; no voters:
I20260806 22:34:41.652370 27796 raft_consensus.cc:2781] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:41.653067 27796 raft_consensus.cc:686] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [term 1 LEADER]: Becoming Leader. State: Replica: 7c99e6636e3f4c1fbad3108217e43897, State: Running, Role: LEADER
I20260806 22:34:41.653614 27686 raft_consensus.cc:2454] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c99e6636e3f4c1fbad3108217e43897 in term 1.
I20260806 22:34:41.653610 27796 consensus_queue.cc:237] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:41.657436 27842 tablet_replica.cc:439] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897: TxnStatusTablet state changed. Reason: New leader 7c99e6636e3f4c1fbad3108217e43897. Latest consensus state: current_term: 1 leader_uuid: "7c99e6636e3f4c1fbad3108217e43897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } } }
I20260806 22:34:41.657949 27842 tablet_replica.cc:442] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:34:41.659699 27053 catalog_manager.cc:5495] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c99e6636e3f4c1fbad3108217e43897 (127.15.198.129). New cstate: current_term: 1 leader_uuid: "7c99e6636e3f4c1fbad3108217e43897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } health_report { overall_health: UNKNOWN } } }
I20260806 22:34:41.660077 27845 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:34:41.660337 27845 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:34:41.693133 27833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.682419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:46527 (local address 127.0.0.1:41320)
0806 22:34:41.682934 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:41.682961 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:41.683207 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:41.684158 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:41.684163 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:41.684178 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:34:41.684432 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.684442 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.686386 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.686394 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.688496 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.688520 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.690035 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.690043 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.690347 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.690538 (+ 191us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:34:41.692242 (+ 1704us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:34:41.692277 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:34:41.692385 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 22:34:41.692653 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252}
I20260806 22:34:41.694172 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.682572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41320 (local address 127.15.198.129:46527)
0806 22:34:41.683606 (+ 1034us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:41.683616 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:41.683644 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:34:41.683780 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:41.683787 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:41.683923 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:41.684116 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:34:41.684637 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.686165 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.688751 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.689611 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.689856 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.690865 (+ 1009us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:34:41.691986 (+ 1121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:34:41.692170 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:41.693620 (+ 1450us) server_negotiation.cc:299] Negotiation successful
0806 22:34:41.693758 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":848}
I20260806 22:34:41.700423 27686 raft_consensus.cc:1261] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Refusing update from remote peer 7c99e6636e3f4c1fbad3108217e43897: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:41.701649 27842 consensus_queue.cc:1035] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:41.701851 27491 raft_consensus.cc:1261] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1 [term 1 FOLLOWER]: Refusing update from remote peer 7c99e6636e3f4c1fbad3108217e43897: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:41.702898 27796 consensus_queue.cc:1035] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:41.717351 27811 tablet_replica.cc:439] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1: TxnStatusTablet state changed. Reason: New leader 7c99e6636e3f4c1fbad3108217e43897. Latest consensus state: current_term: 1 leader_uuid: "7c99e6636e3f4c1fbad3108217e43897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } } }
I20260806 22:34:41.717912 27811 tablet_replica.cc:442] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:34:41.720790 27796 tablet_replica.cc:439] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7c99e6636e3f4c1fbad3108217e43897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } } }
I20260806 22:34:41.721253 27796 tablet_replica.cc:442] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:34:41.724818 27811 tablet_replica.cc:439] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7c99e6636e3f4c1fbad3108217e43897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } } }
I20260806 22:34:41.725208 27811 tablet_replica.cc:442] T 039d3b3a379f466a9dcbef39becacf74 P bf501059a592417f8d04320a65d628e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:34:41.734894 27797 tablet_replica.cc:439] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20: TxnStatusTablet state changed. Reason: New leader 7c99e6636e3f4c1fbad3108217e43897. Latest consensus state: current_term: 1 leader_uuid: "7c99e6636e3f4c1fbad3108217e43897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } } }
I20260806 22:34:41.735323 27797 tablet_replica.cc:442] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:34:41.735991 27842 tablet_replica.cc:439] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7c99e6636e3f4c1fbad3108217e43897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } } }
I20260806 22:34:41.736366 27842 tablet_replica.cc:442] T 039d3b3a379f466a9dcbef39becacf74 P 7c99e6636e3f4c1fbad3108217e43897: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:34:41.739617 27797 tablet_replica.cc:439] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7c99e6636e3f4c1fbad3108217e43897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } } }
I20260806 22:34:41.739938 27797 tablet_replica.cc:442] T 039d3b3a379f466a9dcbef39becacf74 P 06011b629c274269b7ec1e8729285b20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:34:41.761277 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.752913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:44759 (local address 127.15.198.129:46527)
0806 22:34:41.753540 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:41.753550 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:41.754053 (+ 503us) server_negotiation.cc:407] Connection header received
0806 22:34:41.754383 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:41.754388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:41.754478 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:41.754607 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:34:41.755479 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.756600 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.758559 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.759409 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.759574 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.760214 (+ 640us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:34:41.760411 (+ 197us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:34:41.760531 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:41.760719 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 22:34:41.760827 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376}
I20260806 22:34:41.761406 27860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.752924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:46527 (local address 127.15.198.130:44759)
0806 22:34:41.753873 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:41.753917 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:41.754178 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:41.754637 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:41.754643 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:41.754658 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:34:41.755255 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.755269 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.756762 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.756770 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.758376 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.758397 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.759624 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.759635 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.759865 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.759973 (+ 108us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:34:41.760535 (+ 562us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:34:41.760557 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 22:34:41.760679 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 22:34:41.760887 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":648,"threads_started":1}
I20260806 22:34:41.779662 27686 raft_consensus.cc:1261] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Refusing update from remote peer bf501059a592417f8d04320a65d628e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:41.779659 27283 raft_consensus.cc:1261] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897 [term 1 FOLLOWER]: Refusing update from remote peer bf501059a592417f8d04320a65d628e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:41.780570 27795 consensus_queue.cc:1035] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:41.781643 27811 consensus_queue.cc:1035] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:41.808522 27804 mvcc.cc:205] Tried to move back new op lower bound from 7315684072560840704 to 7315684071410049024. Current Snapshot: MvccSnapshot[applied={T|T < 7315684072560840704}]
I20260806 22:34:41.877285 27780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.856770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:57999 (local address 127.15.198.130:39681)
0806 22:34:41.857053 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:41.857061 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:41.865232 (+ 8171us) server_negotiation.cc:407] Connection header received
0806 22:34:41.865901 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:41.865909 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:41.866027 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:41.866108 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:34:41.867184 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.868537 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.872057 (+ 3520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.873544 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.873904 (+ 360us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.875437 (+ 1533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:34:41.875744 (+ 307us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:34:41.875936 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:41.876367 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 22:34:41.876576 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260806 22:34:41.877610 27868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:41.857261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:39681 (local address 127.15.198.129:57999)
0806 22:34:41.865073 (+ 7812us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:41.865102 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:41.865651 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:41.866354 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:41.866358 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:41.866373 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:34:41.866856 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.866881 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.868726 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.868732 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.871608 (+ 2876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:41.871641 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:41.874234 (+ 2593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:41.874247 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:41.874709 (+ 462us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:41.874911 (+ 202us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:34:41.876037 (+ 1126us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:34:41.876061 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 22:34:41.876218 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 22:34:41.876575 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7468,"thread_start_us":7993,"threads_started":1}
I20260806 22:34:42.002096 27052 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:34:42.002914 27052 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:34:42.013685 27052 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745: Sending DeleteTablet for 3 replicas of tablet 8ac44d59ebfa444f8d0e3e7cab6f6984
I20260806 22:34:42.016299 27251 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ac44d59ebfa444f8d0e3e7cab6f6984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:42 UTC) from {username='oryx', principal='oryx/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:41304
I20260806 22:34:42.020237 27461 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ac44d59ebfa444f8d0e3e7cab6f6984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:42 UTC) from {username='oryx', principal='oryx/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:53512
I20260806 22:34:42.030087 27655 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ac44d59ebfa444f8d0e3e7cab6f6984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:42 UTC) from {username='oryx', principal='oryx/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:56678
I20260806 22:34:42.065097 27873 tablet_replica.cc:330] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20: stopping tablet replica
I20260806 22:34:42.065948 27873 raft_consensus.cc:2229] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:34:42.066871 27873 raft_consensus.cc:2258] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:42.070691 27873 ts_tablet_manager.cc:1905] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:42.073940 27872 tablet_replica.cc:330] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1: stopping tablet replica
I20260806 22:34:42.074643 27872 raft_consensus.cc:2229] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:34:42.074761 27870 tablet_replica.cc:330] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897: stopping tablet replica
I20260806 22:34:42.075273 27870 raft_consensus.cc:2229] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:34:42.075870 27870 raft_consensus.cc:2258] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:42.075845 27872 raft_consensus.cc:2258] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:42.079190 27872 ts_tablet_manager.cc:1905] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:42.086743 27870 ts_tablet_manager.cc:1905] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:42.105551 27872 ts_tablet_manager.cc:1918] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:34:42.105822 27870 ts_tablet_manager.cc:1918] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:34:42.106237 27870 log.cc:1196] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/8ac44d59ebfa444f8d0e3e7cab6f6984
I20260806 22:34:42.107752 27870 ts_tablet_manager.cc:1939] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 7c99e6636e3f4c1fbad3108217e43897: Deleting consensus metadata
I20260806 22:34:42.111887 27040 catalog_manager.cc:4841] TS 7c99e6636e3f4c1fbad3108217e43897 (127.15.198.129:46527): tablet 8ac44d59ebfa444f8d0e3e7cab6f6984 (table test-table [id=7003bd83e1004d90a23c02f8d8d6436c]) successfully deleted
I20260806 22:34:42.113415 27872 log.cc:1196] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/8ac44d59ebfa444f8d0e3e7cab6f6984
I20260806 22:34:42.116905 27872 ts_tablet_manager.cc:1939] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P bf501059a592417f8d04320a65d628e1: Deleting consensus metadata
I20260806 22:34:42.119303 27873 ts_tablet_manager.cc:1918] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:34:42.119653 27873 log.cc:1196] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/8ac44d59ebfa444f8d0e3e7cab6f6984
I20260806 22:34:42.120398 27036 catalog_manager.cc:4841] TS bf501059a592417f8d04320a65d628e1 (127.15.198.130:39681): tablet 8ac44d59ebfa444f8d0e3e7cab6f6984 (table test-table [id=7003bd83e1004d90a23c02f8d8d6436c]) successfully deleted
I20260806 22:34:42.120744 27873 ts_tablet_manager.cc:1939] T 8ac44d59ebfa444f8d0e3e7cab6f6984 P 06011b629c274269b7ec1e8729285b20: Deleting consensus metadata
I20260806 22:34:42.124956 27040 catalog_manager.cc:4841] TS 06011b629c274269b7ec1e8729285b20 (127.15.198.131:41311): tablet 8ac44d59ebfa444f8d0e3e7cab6f6984 (table test-table [id=7003bd83e1004d90a23c02f8d8d6436c]) successfully deleted
Aug 06 22:34:42 dist-test-slave-h3ft krb5kdc[27000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260806 22:34:42.207204 16154 mini_kdc.cc:317] Time spent kinit for test-admin: real 0.159s user 0.017s sys 0.035s
Aug 06 22:34:42 dist-test-slave-h3ft krb5kdc[27000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:42 dist-test-slave-h3ft krb5kdc[27000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:34:42.258764 27884 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:42.215272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60474 (local address 127.15.198.190:40953)
0806 22:34:42.232077 (+ 16805us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:42.232262 (+ 185us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:42.232337 (+ 75us) server_negotiation.cc:407] Connection header received
0806 22:34:42.232483 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:42.232494 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:42.232771 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:42.233012 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:42.236478 (+ 3466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:42.237783 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:42.240720 (+ 2937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:42.241493 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:42.241803 (+ 310us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:42.257833 (+ 16030us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60474: BlockingRecv error: recv got EOF from 127.0.0.1:60474 (error 108)
Metrics: {"server-negotiator.queue_time_us":16476,"thread_start_us":16832,"threads_started":1}
Aug 06 22:34:42 dist-test-slave-h3ft krb5kdc[27000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM
I20260806 22:34:42.323498 27884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:42.270102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60490 (local address 127.15.198.190:40953)
0806 22:34:42.270591 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:42.270599 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:42.278587 (+ 7988us) server_negotiation.cc:407] Connection header received
0806 22:34:42.279746 (+ 1159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:42.279756 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:42.279976 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:42.280161 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:42.284676 (+ 4515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:42.286932 (+ 2256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:42.290802 (+ 3870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:42.292287 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:42.292513 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:42.304885 (+ 12372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:42.304919 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:42.304939 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:42.305092 (+ 153us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:42.313541 (+ 8449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:42.316223 (+ 2682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:42.316243 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:42.316258 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:42.316666 (+ 408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:42.317724 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:42.317732 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:42.317736 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:42.318618 (+ 882us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:42.319263 (+ 645us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:42.321952 (+ 2689us) server_negotiation.cc:299] Negotiation successful
0806 22:34:42.322591 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260806 22:34:42.334544 27052 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:34:42.336398 27052 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:34:42.363389 27251 tablet_service.cc:1467] Processing CreateTablet for tablet 88e56101dfba405d9636690ba868f744 (DEFAULT_TABLE table=test-table [id=7843b6f218d94ade90592ed73783ff90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:42.363611 27461 tablet_service.cc:1467] Processing CreateTablet for tablet 88e56101dfba405d9636690ba868f744 (DEFAULT_TABLE table=test-table [id=7843b6f218d94ade90592ed73783ff90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:42.364550 27251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88e56101dfba405d9636690ba868f744. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:42.364547 27461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88e56101dfba405d9636690ba868f744. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:42.369863 27655 tablet_service.cc:1467] Processing CreateTablet for tablet 88e56101dfba405d9636690ba868f744 (DEFAULT_TABLE table=test-table [id=7843b6f218d94ade90592ed73783ff90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:42.370945 27655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88e56101dfba405d9636690ba868f744. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:42.389235 27893 tablet_bootstrap.cc:492] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897: Bootstrap starting.
I20260806 22:34:42.394156 27896 tablet_bootstrap.cc:492] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20: Bootstrap starting.
I20260806 22:34:42.401882 27897 tablet_bootstrap.cc:492] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1: Bootstrap starting.
I20260806 22:34:42.402887 27893 tablet_bootstrap.cc:654] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:42.404372 27896 tablet_bootstrap.cc:654] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:42.407689 27897 tablet_bootstrap.cc:654] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:42.416929 27896 tablet_bootstrap.cc:492] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20: No bootstrap required, opened a new log
I20260806 22:34:42.417390 27896 ts_tablet_manager.cc:1397] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20: Time spent bootstrapping tablet: real 0.024s user 0.011s sys 0.004s
I20260806 22:34:42.419780 27896 raft_consensus.cc:348] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.420486 27896 raft_consensus.cc:374] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:42.420727 27896 raft_consensus.cc:729] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06011b629c274269b7ec1e8729285b20, State: Initialized, Role: FOLLOWER
I20260806 22:34:42.423246 27897 tablet_bootstrap.cc:492] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1: No bootstrap required, opened a new log
I20260806 22:34:42.423322 27893 tablet_bootstrap.cc:492] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897: No bootstrap required, opened a new log
I20260806 22:34:42.423954 27897 ts_tablet_manager.cc:1397] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1: Time spent bootstrapping tablet: real 0.022s user 0.014s sys 0.004s
I20260806 22:34:42.426290 27897 raft_consensus.cc:348] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.427047 27897 raft_consensus.cc:374] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:42.427380 27897 raft_consensus.cc:729] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf501059a592417f8d04320a65d628e1, State: Initialized, Role: FOLLOWER
I20260806 22:34:42.428063 27893 ts_tablet_manager.cc:1397] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897: Time spent bootstrapping tablet: real 0.039s user 0.012s sys 0.003s
I20260806 22:34:42.428079 27897 consensus_queue.cc:260] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.428676 27896 consensus_queue.cc:260] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.429728 27905 raft_consensus.cc:482] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:34:42.430162 27905 raft_consensus.cc:504] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.430294 27893 raft_consensus.cc:348] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.430970 27893 raft_consensus.cc:374] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:42.431229 27893 raft_consensus.cc:729] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c99e6636e3f4c1fbad3108217e43897, State: Initialized, Role: FOLLOWER
I20260806 22:34:42.433327 27897 ts_tablet_manager.cc:1428] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1: Time spent starting tablet: real 0.009s user 0.001s sys 0.004s
I20260806 22:34:42.431952 27893 consensus_queue.cc:260] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.449374 27893 ts_tablet_manager.cc:1428] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897: Time spent starting tablet: real 0.021s user 0.001s sys 0.004s
I20260806 22:34:42.451576 27896 ts_tablet_manager.cc:1428] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20: Time spent starting tablet: real 0.034s user 0.021s sys 0.008s
I20260806 22:34:42.462885 27905 leader_election.cc:288] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c99e6636e3f4c1fbad3108217e43897 (127.15.198.129:46527), bf501059a592417f8d04320a65d628e1 (127.15.198.130:39681)
I20260806 22:34:42.481364 27908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:42.436017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:46527 (local address 127.15.198.131:50861)
0806 22:34:42.461369 (+ 25352us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:42.461445 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:42.461919 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:42.464541 (+ 2622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:42.464551 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:42.464661 (+ 110us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:42.466458 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:42.466508 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:42.469673 (+ 3165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:42.469693 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:42.474565 (+ 4872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:42.474606 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:42.478825 (+ 4219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:42.478846 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:42.479413 (+ 567us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:42.479499 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 22:34:42.479717 (+ 218us) client_negotiation.cc:240] Negotiation successful
0806 22:34:42.480133 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24961,"thread_start_us":24650,"threads_started":1}
I20260806 22:34:42.483557 27909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:42.436983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:50861 (local address 127.15.198.129:46527)
0806 22:34:42.463289 (+ 26306us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:42.463316 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:42.463367 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:34:42.463602 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:42.463611 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:42.463794 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:42.464065 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:42.466936 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:42.469263 (+ 2327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:42.475023 (+ 5760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:42.477618 (+ 2595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:42.478226 (+ 608us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:42.478432 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:42.482247 (+ 3815us) server_negotiation.cc:299] Negotiation successful
0806 22:34:42.482682 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25968,"thread_start_us":24504,"threads_started":1}
I20260806 22:34:42.486150 27281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e56101dfba405d9636690ba868f744" candidate_uuid: "06011b629c274269b7ec1e8729285b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c99e6636e3f4c1fbad3108217e43897" is_pre_election: true
I20260806 22:34:42.486937 27281 raft_consensus.cc:2454] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06011b629c274269b7ec1e8729285b20 in term 0.
I20260806 22:34:42.488267 27592 leader_election.cc:302] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06011b629c274269b7ec1e8729285b20, 7c99e6636e3f4c1fbad3108217e43897; no voters:
I20260806 22:34:42.489578 27905 raft_consensus.cc:2781] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:34:42.489885 27905 raft_consensus.cc:482] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:34:42.490094 27905 raft_consensus.cc:3037] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:42.497030 27910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:42.454450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:39681 (local address 127.15.198.131:53193)
0806 22:34:42.476495 (+ 22045us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:42.476543 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:42.476802 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:42.481001 (+ 4199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:42.481008 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:42.481063 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:34:42.482060 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:42.482092 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:42.484763 (+ 2671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:42.484769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:42.490293 (+ 5524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:42.490326 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:42.494685 (+ 4359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:42.494696 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:42.495338 (+ 642us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:42.495487 (+ 149us) client_negotiation.cc:769] Sending connection context
0806 22:34:42.495700 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 22:34:42.496076 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21769,"thread_start_us":21481,"threads_started":1}
I20260806 22:34:42.496917 27905 raft_consensus.cc:504] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.497771 27800 raft_consensus.cc:482] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:34:42.498057 27800 raft_consensus.cc:504] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.498608 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:42.455411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:53193 (local address 127.15.198.130:39681)
0806 22:34:42.479546 (+ 24135us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:42.479567 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:42.479618 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:34:42.479804 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:42.479833 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:42.480056 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:42.480310 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:34:42.482508 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:42.484398 (+ 1890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:42.490714 (+ 6316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:42.493409 (+ 2695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:42.493907 (+ 498us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:42.494320 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:42.496688 (+ 2368us) server_negotiation.cc:299] Negotiation successful
0806 22:34:42.497038 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23826,"thread_start_us":25184,"threads_started":1}
I20260806 22:34:42.499557 27800 leader_election.cc:288] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c99e6636e3f4c1fbad3108217e43897 (127.15.198.129:46527), 06011b629c274269b7ec1e8729285b20 (127.15.198.131:41311)
I20260806 22:34:42.500195 27491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e56101dfba405d9636690ba868f744" candidate_uuid: "06011b629c274269b7ec1e8729285b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf501059a592417f8d04320a65d628e1" is_pre_election: true
I20260806 22:34:42.500763 27491 raft_consensus.cc:2454] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06011b629c274269b7ec1e8729285b20 in term 0.
I20260806 22:34:42.500818 27490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e56101dfba405d9636690ba868f744" candidate_uuid: "06011b629c274269b7ec1e8729285b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf501059a592417f8d04320a65d628e1"
I20260806 22:34:42.501338 27490 raft_consensus.cc:3037] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:42.501991 27686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e56101dfba405d9636690ba868f744" candidate_uuid: "bf501059a592417f8d04320a65d628e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06011b629c274269b7ec1e8729285b20" is_pre_election: true
I20260806 22:34:42.502943 27686 raft_consensus.cc:2379] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bf501059a592417f8d04320a65d628e1 in current term 1: Already voted for candidate 06011b629c274269b7ec1e8729285b20 in this term.
I20260806 22:34:42.503394 27281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e56101dfba405d9636690ba868f744" candidate_uuid: "bf501059a592417f8d04320a65d628e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c99e6636e3f4c1fbad3108217e43897" is_pre_election: true
I20260806 22:34:42.504069 27281 raft_consensus.cc:2454] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf501059a592417f8d04320a65d628e1 in term 0.
I20260806 22:34:42.505596 27390 leader_election.cc:302] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c99e6636e3f4c1fbad3108217e43897, bf501059a592417f8d04320a65d628e1; no voters: 06011b629c274269b7ec1e8729285b20
I20260806 22:34:42.506106 27281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e56101dfba405d9636690ba868f744" candidate_uuid: "06011b629c274269b7ec1e8729285b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c99e6636e3f4c1fbad3108217e43897"
I20260806 22:34:42.506572 27281 raft_consensus.cc:3037] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:42.498692 27905 leader_election.cc:288] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [CANDIDATE]: Term 1 election: Requested vote from peers 7c99e6636e3f4c1fbad3108217e43897 (127.15.198.129:46527), bf501059a592417f8d04320a65d628e1 (127.15.198.130:39681)
I20260806 22:34:42.509584 27490 raft_consensus.cc:2454] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06011b629c274269b7ec1e8729285b20 in term 1.
I20260806 22:34:42.510412 27800 raft_consensus.cc:2741] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260806 22:34:42.511109 27587 leader_election.cc:302] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06011b629c274269b7ec1e8729285b20, bf501059a592417f8d04320a65d628e1; no voters:
I20260806 22:34:42.512063 27905 raft_consensus.cc:2781] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:42.513270 27905 raft_consensus.cc:686] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [term 1 LEADER]: Becoming Leader. State: Replica: 06011b629c274269b7ec1e8729285b20, State: Running, Role: LEADER
I20260806 22:34:42.514348 27905 consensus_queue.cc:237] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.515367 27281 raft_consensus.cc:2454] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06011b629c274269b7ec1e8729285b20 in term 1.
I20260806 22:34:42.523093 27052 catalog_manager.cc:5495] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06011b629c274269b7ec1e8729285b20 (127.15.198.131). New cstate: current_term: 1 leader_uuid: "06011b629c274269b7ec1e8729285b20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } health_report { overall_health: HEALTHY } } }
I20260806 22:34:42.569032 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:42.552239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56694 (local address 127.15.198.131:41311)
0806 22:34:42.553587 (+ 1348us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:42.553598 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:42.553648 (+ 50us) server_negotiation.cc:407] Connection header received
0806 22:34:42.553937 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:42.553944 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:42.554155 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:42.554617 (+ 462us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:34:42.555945 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:42.558188 (+ 2243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:42.562171 (+ 3983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:42.563561 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:42.563826 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:42.565204 (+ 1378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:34:42.567061 (+ 1857us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:34:42.567483 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:42.567768 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 22:34:42.567992 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1058,"thread_start_us":760,"threads_started":1}
I20260806 22:34:42.580243 27281 raft_consensus.cc:1261] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897 [term 1 FOLLOWER]: Refusing update from remote peer 06011b629c274269b7ec1e8729285b20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:42.580386 27490 raft_consensus.cc:1261] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1 [term 1 FOLLOWER]: Refusing update from remote peer 06011b629c274269b7ec1e8729285b20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:42.582000 27905 consensus_queue.cc:1035] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:42.582713 27906 consensus_queue.cc:1035] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:42.658970 27053 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:34:42.659588 27053 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:34:42.667150 27053 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745: Sending DeleteTablet for 3 replicas of tablet 88e56101dfba405d9636690ba868f744
I20260806 22:34:42.668982 27251 tablet_service.cc:1514] Processing DeleteTablet for tablet 88e56101dfba405d9636690ba868f744 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:42 UTC) from {username='oryx', principal='oryx/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:41304
I20260806 22:34:42.670739 27461 tablet_service.cc:1514] Processing DeleteTablet for tablet 88e56101dfba405d9636690ba868f744 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:42 UTC) from {username='oryx', principal='oryx/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:53512
I20260806 22:34:42.675253 27655 tablet_service.cc:1514] Processing DeleteTablet for tablet 88e56101dfba405d9636690ba868f744 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:42 UTC) from {username='oryx', principal='oryx/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:56678
I20260806 22:34:42.681282 27053 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:34:42.683234 27053 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:34:42.689438 27929 tablet_replica.cc:330] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1: stopping tablet replica
I20260806 22:34:42.691684 27929 raft_consensus.cc:2229] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:34:42.692534 27929 raft_consensus.cc:2258] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:42.705129 27929 ts_tablet_manager.cc:1905] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:42.709654 27250 tablet_service.cc:1467] Processing CreateTablet for tablet 1cc54e6def254005972211c6b647acdf (DEFAULT_TABLE table=test-table [id=18f1094c11484fcaa1e613f167fb78ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:42.713815 27461 tablet_service.cc:1467] Processing CreateTablet for tablet 1cc54e6def254005972211c6b647acdf (DEFAULT_TABLE table=test-table [id=18f1094c11484fcaa1e613f167fb78ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:42.714977 27461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cc54e6def254005972211c6b647acdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:42.721361 27250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cc54e6def254005972211c6b647acdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:42.722299 27654 tablet_service.cc:1467] Processing CreateTablet for tablet 1cc54e6def254005972211c6b647acdf (DEFAULT_TABLE table=test-table [id=18f1094c11484fcaa1e613f167fb78ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:34:42.723649 27654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cc54e6def254005972211c6b647acdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:42.732466 27929 ts_tablet_manager.cc:1918] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:34:42.733023 27929 log.cc:1196] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/88e56101dfba405d9636690ba868f744
I20260806 22:34:42.734207 27929 ts_tablet_manager.cc:1939] T 88e56101dfba405d9636690ba868f744 P bf501059a592417f8d04320a65d628e1: Deleting consensus metadata
I20260806 22:34:42.750432 27036 catalog_manager.cc:4841] TS bf501059a592417f8d04320a65d628e1 (127.15.198.130:39681): tablet 88e56101dfba405d9636690ba868f744 (table test-table [id=7843b6f218d94ade90592ed73783ff90]) successfully deleted
I20260806 22:34:42.763841 27930 tablet_replica.cc:330] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20: stopping tablet replica
I20260806 22:34:42.764670 27930 raft_consensus.cc:2229] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:34:42.768167 27928 tablet_replica.cc:330] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897: stopping tablet replica
I20260806 22:34:42.768672 27928 raft_consensus.cc:2229] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:34:42.771484 27897 tablet_bootstrap.cc:492] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1: Bootstrap starting.
I20260806 22:34:42.771741 27928 raft_consensus.cc:2258] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:42.773352 27930 raft_consensus.cc:2258] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:42.775933 27897 tablet_bootstrap.cc:654] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:42.776269 27893 tablet_bootstrap.cc:492] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897: Bootstrap starting.
I20260806 22:34:42.777271 27896 tablet_bootstrap.cc:492] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20: Bootstrap starting.
I20260806 22:34:42.778226 27928 ts_tablet_manager.cc:1905] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:42.782727 27893 tablet_bootstrap.cc:654] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:42.785790 27897 tablet_bootstrap.cc:492] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1: No bootstrap required, opened a new log
I20260806 22:34:42.786109 27896 tablet_bootstrap.cc:654] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20: Neither blocks nor log segments found. Creating new log.
I20260806 22:34:42.786486 27897 ts_tablet_manager.cc:1397] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.001s
I20260806 22:34:42.788694 27897 raft_consensus.cc:348] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.789808 27897 raft_consensus.cc:374] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:42.790060 27897 raft_consensus.cc:729] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf501059a592417f8d04320a65d628e1, State: Initialized, Role: FOLLOWER
I20260806 22:34:42.790536 27930 ts_tablet_manager.cc:1905] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:42.795770 27928 ts_tablet_manager.cc:1918] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:34:42.796064 27928 log.cc:1196] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/88e56101dfba405d9636690ba868f744
I20260806 22:34:42.797135 27928 ts_tablet_manager.cc:1939] T 88e56101dfba405d9636690ba868f744 P 7c99e6636e3f4c1fbad3108217e43897: Deleting consensus metadata
I20260806 22:34:42.790843 27897 consensus_queue.cc:260] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.800485 27040 catalog_manager.cc:4841] TS 7c99e6636e3f4c1fbad3108217e43897 (127.15.198.129:46527): tablet 88e56101dfba405d9636690ba868f744 (table test-table [id=7843b6f218d94ade90592ed73783ff90]) successfully deleted
I20260806 22:34:42.808243 27897 ts_tablet_manager.cc:1428] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1: Time spent starting tablet: real 0.021s user 0.001s sys 0.003s
I20260806 22:34:42.810338 27896 tablet_bootstrap.cc:492] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20: No bootstrap required, opened a new log
I20260806 22:34:42.810899 27896 ts_tablet_manager.cc:1397] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20: Time spent bootstrapping tablet: real 0.043s user 0.006s sys 0.008s
I20260806 22:34:42.813498 27896 raft_consensus.cc:348] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.814342 27896 raft_consensus.cc:374] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:42.814592 27896 raft_consensus.cc:729] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06011b629c274269b7ec1e8729285b20, State: Initialized, Role: FOLLOWER
I20260806 22:34:42.816761 27896 consensus_queue.cc:260] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.818783 27893 tablet_bootstrap.cc:492] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897: No bootstrap required, opened a new log
I20260806 22:34:42.819758 27896 ts_tablet_manager.cc:1428] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20: Time spent starting tablet: real 0.008s user 0.002s sys 0.003s
I20260806 22:34:42.820287 27893 ts_tablet_manager.cc:1397] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897: Time spent bootstrapping tablet: real 0.044s user 0.017s sys 0.008s
I20260806 22:34:42.822616 27893 raft_consensus.cc:348] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.823472 27893 raft_consensus.cc:374] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:42.823777 27893 raft_consensus.cc:729] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c99e6636e3f4c1fbad3108217e43897, State: Initialized, Role: FOLLOWER
I20260806 22:34:42.827975 27930 ts_tablet_manager.cc:1918] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:34:42.828361 27930 log.cc:1196] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/88e56101dfba405d9636690ba868f744
I20260806 22:34:42.829375 27930 ts_tablet_manager.cc:1939] T 88e56101dfba405d9636690ba868f744 P 06011b629c274269b7ec1e8729285b20: Deleting consensus metadata
I20260806 22:34:42.824565 27893 consensus_queue.cc:260] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.831683 27040 catalog_manager.cc:4841] TS 06011b629c274269b7ec1e8729285b20 (127.15.198.131:41311): tablet 88e56101dfba405d9636690ba868f744 (table test-table [id=7843b6f218d94ade90592ed73783ff90]) successfully deleted
I20260806 22:34:42.834390 27893 ts_tablet_manager.cc:1428] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260806 22:34:42.870302 27842 raft_consensus.cc:482] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:34:42.870900 27842 raft_consensus.cc:504] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.872686 27842 leader_election.cc:288] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf501059a592417f8d04320a65d628e1 (127.15.198.130:39681), 06011b629c274269b7ec1e8729285b20 (127.15.198.131:41311)
I20260806 22:34:42.873960 27491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cc54e6def254005972211c6b647acdf" candidate_uuid: "7c99e6636e3f4c1fbad3108217e43897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf501059a592417f8d04320a65d628e1" is_pre_election: true
I20260806 22:34:42.874285 27686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cc54e6def254005972211c6b647acdf" candidate_uuid: "7c99e6636e3f4c1fbad3108217e43897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06011b629c274269b7ec1e8729285b20" is_pre_election: true
I20260806 22:34:42.874605 27491 raft_consensus.cc:2454] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c99e6636e3f4c1fbad3108217e43897 in term 0.
I20260806 22:34:42.874931 27686 raft_consensus.cc:2454] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c99e6636e3f4c1fbad3108217e43897 in term 0.
I20260806 22:34:42.875703 27181 leader_election.cc:302] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c99e6636e3f4c1fbad3108217e43897, bf501059a592417f8d04320a65d628e1; no voters:
I20260806 22:34:42.876453 27842 raft_consensus.cc:2781] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:34:42.876708 27842 raft_consensus.cc:482] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:34:42.876996 27842 raft_consensus.cc:3037] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:42.887773 27842 raft_consensus.cc:504] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.890393 27491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cc54e6def254005972211c6b647acdf" candidate_uuid: "7c99e6636e3f4c1fbad3108217e43897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf501059a592417f8d04320a65d628e1"
I20260806 22:34:42.890414 27686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cc54e6def254005972211c6b647acdf" candidate_uuid: "7c99e6636e3f4c1fbad3108217e43897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06011b629c274269b7ec1e8729285b20"
I20260806 22:34:42.890944 27491 raft_consensus.cc:3037] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:42.891038 27686 raft_consensus.cc:3037] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:42.897505 27686 raft_consensus.cc:2454] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c99e6636e3f4c1fbad3108217e43897 in term 1.
I20260806 22:34:42.897862 27491 raft_consensus.cc:2454] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c99e6636e3f4c1fbad3108217e43897 in term 1.
I20260806 22:34:42.899230 27181 leader_election.cc:302] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c99e6636e3f4c1fbad3108217e43897, bf501059a592417f8d04320a65d628e1; no voters:
I20260806 22:34:42.900302 27935 raft_consensus.cc:2781] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:42.903400 27842 leader_election.cc:288] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [CANDIDATE]: Term 1 election: Requested vote from peers bf501059a592417f8d04320a65d628e1 (127.15.198.130:39681), 06011b629c274269b7ec1e8729285b20 (127.15.198.131:41311)
I20260806 22:34:42.903379 27935 raft_consensus.cc:686] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [term 1 LEADER]: Becoming Leader. State: Replica: 7c99e6636e3f4c1fbad3108217e43897, State: Running, Role: LEADER
I20260806 22:34:42.905061 27935 consensus_queue.cc:237] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } }
I20260806 22:34:42.912415 27052 catalog_manager.cc:5495] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c99e6636e3f4c1fbad3108217e43897 (127.15.198.129). New cstate: current_term: 1 leader_uuid: "7c99e6636e3f4c1fbad3108217e43897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c99e6636e3f4c1fbad3108217e43897" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 } health_report { overall_health: UNKNOWN } } }
I20260806 22:34:42.976847 27909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:42.965144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41324 (local address 127.15.198.129:46527)
0806 22:34:42.965601 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:42.965611 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:42.965660 (+ 49us) server_negotiation.cc:407] Connection header received
0806 22:34:42.965992 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:42.965999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:42.966170 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:42.966370 (+ 200us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:34:42.967261 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:42.969255 (+ 1994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:42.972109 (+ 2854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:42.973701 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:42.973959 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:42.975215 (+ 1256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:34:42.975552 (+ 337us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:34:42.975745 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:42.976022 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 22:34:42.976229 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193}
I20260806 22:34:43.011359 27909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:42.997681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:38263 (local address 127.15.198.129:46527)
0806 22:34:42.998230 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:42.998266 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:42.999646 (+ 1380us) server_negotiation.cc:407] Connection header received
0806 22:34:43.000126 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:43.000132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:43.000301 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:43.000567 (+ 266us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:34:43.001780 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:43.003919 (+ 2139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:43.007259 (+ 3340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:43.008153 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:43.008468 (+ 315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:43.009264 (+ 796us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:34:43.009513 (+ 249us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:34:43.009742 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:43.010062 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 22:34:43.010240 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":14,"server-negotiator.queue_time_us":195}
I20260806 22:34:43.011461 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:42.997781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:46527 (local address 127.15.198.129:38263)
0806 22:34:42.999446 (+ 1665us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:42.999479 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:42.999829 (+ 350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:43.000579 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:43.000584 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:43.000620 (+ 36us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:34:43.001452 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:43.001475 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:43.004153 (+ 2678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:43.004172 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:43.006857 (+ 2685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:43.006885 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:43.008556 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:43.008567 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:43.008843 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:43.008932 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:34:43.009712 (+ 780us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:34:43.009747 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:34:43.010007 (+ 260us) client_negotiation.cc:240] Negotiation successful
0806 22:34:43.010313 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1302,"mutex_wait_us":65,"thread_start_us":928,"threads_started":1}
I20260806 22:34:43.037354 27687 raft_consensus.cc:1261] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Refusing update from remote peer 7c99e6636e3f4c1fbad3108217e43897: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:43.040583 27842 consensus_queue.cc:1035] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06011b629c274269b7ec1e8729285b20" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 41311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:43.042052 27491 raft_consensus.cc:1261] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1 [term 1 FOLLOWER]: Refusing update from remote peer 7c99e6636e3f4c1fbad3108217e43897: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:34:43.043326 27842 consensus_queue.cc:1035] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf501059a592417f8d04320a65d628e1" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:34:43.208714 27053 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:34:43.209425 27053 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:34:43.218132 27053 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39467d156c884eadb5f658a1466a5745: Sending DeleteTablet for 3 replicas of tablet 1cc54e6def254005972211c6b647acdf
I20260806 22:34:43.220242 27251 tablet_service.cc:1514] Processing DeleteTablet for tablet 1cc54e6def254005972211c6b647acdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:43 UTC) from {username='oryx', principal='oryx/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:41304
I20260806 22:34:43.220502 27461 tablet_service.cc:1514] Processing DeleteTablet for tablet 1cc54e6def254005972211c6b647acdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:43 UTC) from {username='oryx', principal='oryx/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:53512
I20260806 22:34:43.221292 27928 tablet_replica.cc:330] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897: stopping tablet replica
I20260806 22:34:43.221829 27928 raft_consensus.cc:2229] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:34:43.223193 27928 raft_consensus.cc:2258] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:43.223354 27929 tablet_replica.cc:330] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1: stopping tablet replica
I20260806 22:34:43.223850 27929 raft_consensus.cc:2229] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:34:43.224313 27929 raft_consensus.cc:2258] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:43.226033 27654 tablet_service.cc:1514] Processing DeleteTablet for tablet 1cc54e6def254005972211c6b647acdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:34:43 UTC) from {username='oryx', principal='oryx/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:56678
I20260806 22:34:43.226146 27928 ts_tablet_manager.cc:1905] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:43.226722 27930 tablet_replica.cc:330] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20: stopping tablet replica
I20260806 22:34:43.227216 27930 raft_consensus.cc:2229] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:34:43.227711 27930 raft_consensus.cc:2258] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:34:43.228894 27929 ts_tablet_manager.cc:1905] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:43.232118 27930 ts_tablet_manager.cc:1905] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:34:43.235648 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 27160
I20260806 22:34:43.241705 27929 ts_tablet_manager.cc:1918] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:34:43.242048 27929 log.cc:1196] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/1cc54e6def254005972211c6b647acdf
I20260806 22:34:43.242636 27929 ts_tablet_manager.cc:1939] T 1cc54e6def254005972211c6b647acdf P bf501059a592417f8d04320a65d628e1: Deleting consensus metadata
I20260806 22:34:43.244293 27036 catalog_manager.cc:4841] TS bf501059a592417f8d04320a65d628e1 (127.15.198.130:39681): tablet 1cc54e6def254005972211c6b647acdf (table test-table [id=18f1094c11484fcaa1e613f167fb78ea]) successfully deleted
I20260806 22:34:43.253659 27928 ts_tablet_manager.cc:1918] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:34:43.254009 27928 log.cc:1196] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/1cc54e6def254005972211c6b647acdf
I20260806 22:34:43.254693 27928 ts_tablet_manager.cc:1939] T 1cc54e6def254005972211c6b647acdf P 7c99e6636e3f4c1fbad3108217e43897: Deleting consensus metadata
I20260806 22:34:43.256626 27040 catalog_manager.cc:4841] TS 7c99e6636e3f4c1fbad3108217e43897 (127.15.198.129:46527): tablet 1cc54e6def254005972211c6b647acdf (table test-table [id=18f1094c11484fcaa1e613f167fb78ea]) successfully deleted
I20260806 22:34:43.260159 27930 ts_tablet_manager.cc:1918] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:34:43.260411 27930 log.cc:1196] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/1cc54e6def254005972211c6b647acdf
I20260806 22:34:43.260972 27930 ts_tablet_manager.cc:1939] T 1cc54e6def254005972211c6b647acdf P 06011b629c274269b7ec1e8729285b20: Deleting consensus metadata
I20260806 22:34:43.263644 27040 catalog_manager.cc:4841] TS 06011b629c274269b7ec1e8729285b20 (127.15.198.131:41311): tablet 1cc54e6def254005972211c6b647acdf (table test-table [id=18f1094c11484fcaa1e613f167fb78ea]) successfully deleted
Aug 06 22:34:43 dist-test-slave-h3ft krb5kdc[27000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.198.129@KRBTEST.COM
I20260806 22:34:43.295485 27909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:43.251214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41336 (local address 127.15.198.129:46527)
0806 22:34:43.251650 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:43.251658 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:43.252546 (+ 888us) server_negotiation.cc:407] Connection header received
0806 22:34:43.252869 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:43.252889 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:43.253000 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:43.253136 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:43.257205 (+ 4069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:43.258558 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:43.261062 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:43.262136 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:43.262435 (+ 299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:43.276229 (+ 13794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:43.276283 (+ 54us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:43.276327 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:43.276456 (+ 129us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:43.285414 (+ 8958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:43.287803 (+ 2389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:43.287846 (+ 43us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:43.287866 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:43.288242 (+ 376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:43.289279 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:43.289287 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:43.289291 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:43.290655 (+ 1364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:43.291357 (+ 702us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:43.294039 (+ 2682us) server_negotiation.cc:299] Negotiation successful
0806 22:34:43.294619 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247}
I20260806 22:34:43.297712 27350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:43.583781 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 27160
I20260806 22:34:43.678438 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 27364
Aug 06 22:34:43 dist-test-slave-h3ft krb5kdc[27000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.198.130@KRBTEST.COM
I20260806 22:34:43.722867 27951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:43.680234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53534 (local address 127.15.198.130:39681)
0806 22:34:43.681311 (+ 1077us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:43.681326 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:43.681356 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:34:43.681463 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:43.681467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:43.681610 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:43.681807 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:43.685380 (+ 3573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:43.687206 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:43.689933 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:43.690866 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:43.691278 (+ 412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:43.707184 (+ 15906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:43.707228 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:43.707264 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:43.707375 (+ 111us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:43.714784 (+ 7409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:43.717538 (+ 2754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:43.717568 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:43.717570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:43.718022 (+ 452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:43.718896 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:43.718914 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:43.718918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:43.719864 (+ 946us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:43.720636 (+ 772us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:43.721360 (+ 724us) server_negotiation.cc:299] Negotiation successful
0806 22:34:43.721975 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":888,"thread_start_us":661,"threads_started":1}
I20260806 22:34:43.724691 27549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:43.949676 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 27364
I20260806 22:34:44.021522 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 27565
Aug 06 22:34:44 dist-test-slave-h3ft krb5kdc[27000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.198.131@KRBTEST.COM
I20260806 22:34:44.062064 27954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:44.024285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56704 (local address 127.15.198.131:41311)
0806 22:34:44.025541 (+ 1256us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:44.025558 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:44.025604 (+ 46us) server_negotiation.cc:407] Connection header received
0806 22:34:44.025716 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:44.025720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:44.025895 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:44.026229 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:44.029256 (+ 3027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:44.030703 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:44.033151 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:44.034076 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:44.034420 (+ 344us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:44.046813 (+ 12393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:44.046873 (+ 60us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:44.046913 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:44.047072 (+ 159us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:44.054193 (+ 7121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:44.056183 (+ 1990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:44.056207 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:44.056210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:44.056758 (+ 548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:44.057723 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:44.057738 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:44.057742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:44.058729 (+ 987us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:44.059720 (+ 991us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:44.060415 (+ 695us) server_negotiation.cc:299] Negotiation successful
0806 22:34:44.061034 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1048,"thread_start_us":733,"threads_started":1}
I20260806 22:34:44.063664 27741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:44.282140 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 27565
W20260806 22:34:44.346347 27040 connection.cc:535] server connection from 127.15.198.131:40771 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:34:44.347054 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 27020
I20260806 22:34:44.349788 27114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:44.562139 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 27020
2026-08-06T22:34:44Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (8256 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:34:44 dist-test-slave-h3ft krb5kdc[27963](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:34:44 dist-test-slave-h3ft krb5kdc[27963](info): set up 2 sockets
Aug 06 22:34:44 dist-test-slave-h3ft krb5kdc[27963](info): commencing operation
krb5kdc: starting...
W20260806 22:34:46.396782 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.731s user 0.002s sys 0.094s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:34:46 dist-test-slave-h3ft krb5kdc[27963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:34:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:34:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:34:46.760700 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:44345
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:35983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:44345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:47.206148 27977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:47.206727 27977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:47.206929 27977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:47.215685 27977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:34:47.215845 27977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:47.215905 27977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:47.216019 27977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:34:47.216095 27977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:34:47.227948 27977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:35983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:44345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:44345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:47.229835 27977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:47.231560 27977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:47.246042 27986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:47.246229 27988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:47.246191 27985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:47.247977 27977 server_base.cc:1034] running on GCE node
I20260806 22:34:47.249130 27977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:47.251325 27977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:47.252661 27977 hybrid_clock.cc:648] HybridClock initialized: now 1786055687252577 us; error 61 us; skew 500 ppm
I20260806 22:34:47.253240 27977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T22:34:47Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2724 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:34:47 dist-test-slave-h3ft krb5kdc[27996](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:34:47 dist-test-slave-h3ft krb5kdc[27996](info): set up 2 sockets
Aug 06 22:34:47 dist-test-slave-h3ft krb5kdc[27996](info): commencing operation
krb5kdc: starting...
W20260806 22:34:48.717283 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:34:48 dist-test-slave-h3ft krb5kdc[27996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:34:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:34:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:34:49.088373 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:34327
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:41979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:34327
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:49.523123 28015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:49.523591 28015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:49.523777 28015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:49.531751 28015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:34:49.531950 28015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:49.532073 28015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:49.532186 28015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:34:49.532279 28015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:34:49.543768 28015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:41979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:34327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:34327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:49.545799 28015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:49.547765 28015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:49.562795 28028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:49.562834 28024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:49.562747 28023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:49.563920 28015 server_base.cc:1034] running on GCE node
I20260806 22:34:49.565013 28015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:49.567124 28015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:49.568392 28015 hybrid_clock.cc:648] HybridClock initialized: now 1786055689568312 us; error 68 us; skew 500 ppm
I20260806 22:34:49.568994 28015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:49 dist-test-slave-h3ft krb5kdc[27996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055689, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:49.576628 28015 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:34:49.578907 28015 webserver.cc:460] Webserver started at http://127.15.198.190:43233/ using document root <none> and password file <none>
I20260806 22:34:49.579916 28015 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:49.580094 28015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:49.580497 28015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:49.584513 28015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "055e5a87859541429b98661d03745f94"
format_stamp: "Formatted at 2026-08-06 22:34:49 on dist-test-slave-h3ft"
server_key: "8f0d80f6d9fbbf1b591e9ab6ec108bc9"
server_key_iv: "ffc528b74ec668dcd70a4361088cb7db"
server_key_version: "encryptionkey@0"
I20260806 22:34:49.585500 28015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "055e5a87859541429b98661d03745f94"
format_stamp: "Formatted at 2026-08-06 22:34:49 on dist-test-slave-h3ft"
server_key: "8f0d80f6d9fbbf1b591e9ab6ec108bc9"
server_key_iv: "ffc528b74ec668dcd70a4361088cb7db"
server_key_version: "encryptionkey@0"
I20260806 22:34:49.592552 28015 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 22:34:49.597074 28038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:49.598912 28015 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260806 22:34:49.599182 28015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "055e5a87859541429b98661d03745f94"
format_stamp: "Formatted at 2026-08-06 22:34:49 on dist-test-slave-h3ft"
server_key: "8f0d80f6d9fbbf1b591e9ab6ec108bc9"
server_key_iv: "ffc528b74ec668dcd70a4361088cb7db"
server_key_version: "encryptionkey@0"
I20260806 22:34:49.599450 28015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:49.616851 28015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:49.622650 28015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:49.623234 28015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:49.649924 28015 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:34327
I20260806 22:34:49.649928 28111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:34327 every 8 connection(s)
I20260806 22:34:49.652021 28015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:34:49.657224 28112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:34:49.660295 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 28015
I20260806 22:34:49.660585 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:34:49.661533 16154 external_mini_cluster.cc:1428] Setting key a527aadcf3d195317334b09cc63aa1e3
I20260806 22:34:49.673375 28112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94: Bootstrap starting.
I20260806 22:34:49.678622 28112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94: Neither blocks nor log segments found. Creating new log.
Aug 06 22:34:49 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:34:49.680258 28112 log.cc:824] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:49.685006 28112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94: No bootstrap required, opened a new log
I20260806 22:34:49.691533 28115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:49.664492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.15.198.190:34327)
0806 22:34:49.665570 (+ 1078us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:49.665586 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:49.665648 (+ 62us) server_negotiation.cc:407] Connection header received
0806 22:34:49.667742 (+ 2094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:49.667774 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:49.668277 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:49.668743 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:49.671179 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:49.672754 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:49.674688 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:49.675573 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:49.675752 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:49.681632 (+ 5880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:49.681650 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:49.681679 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:49.681745 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:49.685682 (+ 3937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:49.686870 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:49.686880 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:49.686889 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:49.687199 (+ 310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:49.687738 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:49.687742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:49.687745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:49.688549 (+ 804us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:49.688837 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:49.689446 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 22:34:49.689962 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":809,"thread_start_us":504,"threads_started":1}
I20260806 22:34:49.697054 28112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e5a87859541429b98661d03745f94" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 34327 } }
I20260806 22:34:49.697638 28112 raft_consensus.cc:374] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:34:49.697779 28112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 055e5a87859541429b98661d03745f94, State: Initialized, Role: FOLLOWER
I20260806 22:34:49.698531 28112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e5a87859541429b98661d03745f94" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 34327 } }
I20260806 22:34:49.698871 28112 raft_consensus.cc:388] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:34:49.699095 28112 raft_consensus.cc:482] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:34:49.699331 28112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:34:49.702594 28112 raft_consensus.cc:504] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e5a87859541429b98661d03745f94" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 34327 } }
I20260806 22:34:49.703401 28112 leader_election.cc:302] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 055e5a87859541429b98661d03745f94; no voters:
I20260806 22:34:49.704286 28112 leader_election.cc:288] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:34:49.704485 28117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:34:49.705984 28117 raft_consensus.cc:686] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [term 1 LEADER]: Becoming Leader. State: Replica: 055e5a87859541429b98661d03745f94, State: Running, Role: LEADER
I20260806 22:34:49.706670 28117 consensus_queue.cc:237] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e5a87859541429b98661d03745f94" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 34327 } }
I20260806 22:34:49.707300 28112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:34:49.711609 28119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 055e5a87859541429b98661d03745f94. Latest consensus state: current_term: 1 leader_uuid: "055e5a87859541429b98661d03745f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e5a87859541429b98661d03745f94" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 34327 } } }
I20260806 22:34:49.711997 28118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "055e5a87859541429b98661d03745f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e5a87859541429b98661d03745f94" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 34327 } } }
I20260806 22:34:49.712334 28118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:49.712334 28119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:49.715251 28131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:34:49.720853 28131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:34:49.733345 28131 catalog_manager.cc:1347] Generated new cluster ID: b0c321be95e84cf2920f1061b540ead1
I20260806 22:34:49.733530 28131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:34:49.755244 28131 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:34:49.757644 28131 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:34:49.768170 28131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94: Generated new TSK 0
I20260806 22:34:49.769732 28131 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
I20260806 22:34:49.864538 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:34327
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:41979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:50.318737 28150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:50.319233 28150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:50.319422 28150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:50.327831 28150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:50.328038 28150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:50.328219 28150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:34:50.340376 28150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:41979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:34327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:50.342289 28150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:50.344517 28150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:50.361519 28155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:50.361434 28159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:50.361486 28157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:50.362804 28150 server_base.cc:1034] running on GCE node
I20260806 22:34:50.363875 28150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:50.365510 28150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:50.366787 28150 hybrid_clock.cc:648] HybridClock initialized: now 1786055690366703 us; error 81 us; skew 500 ppm
I20260806 22:34:50.367230 28150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:50 dist-test-slave-h3ft krb5kdc[27996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055690, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:50.372381 28150 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:34:50.374290 28150 webserver.cc:460] Webserver started at http://127.15.198.129:38249/ using document root <none> and password file <none>
I20260806 22:34:50.375295 28150 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:50.375427 28150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:50.375804 28150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:50.380036 28150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "3ddc06c44ab34fa59e3cecd7a24dc2c9"
format_stamp: "Formatted at 2026-08-06 22:34:50 on dist-test-slave-h3ft"
server_key: "be36becff6a573a978b5bb1aad0f10c8"
server_key_iv: "d98f6609f8491505225fe182cac8c7e1"
server_key_version: "encryptionkey@0"
I20260806 22:34:50.381023 28150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "3ddc06c44ab34fa59e3cecd7a24dc2c9"
format_stamp: "Formatted at 2026-08-06 22:34:50 on dist-test-slave-h3ft"
server_key: "be36becff6a573a978b5bb1aad0f10c8"
server_key_iv: "d98f6609f8491505225fe182cac8c7e1"
server_key_version: "encryptionkey@0"
I20260806 22:34:50.387706 28150 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 22:34:50.392094 28167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:50.393956 28150 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:34:50.394186 28150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "3ddc06c44ab34fa59e3cecd7a24dc2c9"
format_stamp: "Formatted at 2026-08-06 22:34:50 on dist-test-slave-h3ft"
server_key: "be36becff6a573a978b5bb1aad0f10c8"
server_key_iv: "d98f6609f8491505225fe182cac8c7e1"
server_key_version: "encryptionkey@0"
I20260806 22:34:50.394454 28150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:50.411866 28150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:50.417944 28150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:50.418372 28150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:50.419780 28150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:50.421635 28150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:50.421718 28150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:50.421937 28150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:50.422005 28150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:50.462126 28150 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:44831
I20260806 22:34:50.462177 28330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:44831 every 8 connection(s)
I20260806 22:34:50.464041 28150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:34:50.468116 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 28150
I20260806 22:34:50.468472 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:34:50.469360 16154 external_mini_cluster.cc:1428] Setting key 941c94e5dc8f5983529f913087253ae2
Aug 06 22:34:50 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055690, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:34:50.493922 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:50.467844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:46663 (local address 127.15.198.190:34327)
0806 22:34:50.468624 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:50.468632 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:50.469931 (+ 1299us) server_negotiation.cc:407] Connection header received
0806 22:34:50.472580 (+ 2649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:50.472587 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:50.472711 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:50.472880 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:50.474963 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:50.476762 (+ 1799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:50.478735 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:50.479655 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:50.479849 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:50.485288 (+ 5439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:50.485296 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:50.485299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:50.485354 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:50.488847 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:50.490124 (+ 1277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:50.490136 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:50.490154 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:50.490392 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:50.491249 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:50.491254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:50.491256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:50.491673 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:50.491838 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:50.493257 (+ 1419us) server_negotiation.cc:299] Negotiation successful
0806 22:34:50.493549 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":443,"threads_started":1}
I20260806 22:34:50.495987 28335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:50.468605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:46663)
0806 22:34:50.469684 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:50.469743 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:50.472194 (+ 2451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:50.473190 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:50.473201 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:50.473885 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:50.474714 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:50.474739 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:50.476987 (+ 2248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:50.476992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:50.478532 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:50.478550 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:50.480011 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:50.480053 (+ 42us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:50.480413 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:50.481458 (+ 1045us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:50.481488 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:50.484906 (+ 3418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:50.489124 (+ 4218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:50.489136 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:50.489159 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:50.489912 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:50.490592 (+ 680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:50.490597 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:50.490599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:50.491073 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:50.491875 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:50.491889 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:50.492599 (+ 710us) client_negotiation.cc:769] Sending connection context
0806 22:34:50.493011 (+ 412us) client_negotiation.cc:240] Negotiation successful
0806 22:34:50.493816 (+ 805us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":444,"threads_started":1}
I20260806 22:34:50.498567 28331 heartbeater.cc:347] Connected to a master server at 127.15.198.190:34327
I20260806 22:34:50.499174 28331 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:50.500439 28331 heartbeater.cc:511] Master 127.15.198.190:34327 requested a full tablet report, sending...
I20260806 22:34:50.504115 28063 ts_manager.cc:194] Registered new tserver with Master: 3ddc06c44ab34fa59e3cecd7a24dc2c9 (127.15.198.129:44831)
I20260806 22:34:50.507027 28063 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:46663
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
I20260806 22:34:50.560956 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:34327
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:41979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:51.005332 28338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:51.005704 28338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:51.005878 28338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:51.014118 28338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:51.014263 28338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:51.014498 28338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:34:51.025897 28338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:41979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:34327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:51.027793 28338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:51.029623 28338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:51.045603 28349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:51.045655 28353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:51.045746 28350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:51.047112 28338 server_base.cc:1034] running on GCE node
I20260806 22:34:51.047951 28338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:51.049240 28338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:51.050525 28338 hybrid_clock.cc:648] HybridClock initialized: now 1786055691050462 us; error 60 us; skew 500 ppm
I20260806 22:34:51.050978 28338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:51 dist-test-slave-h3ft krb5kdc[27996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055691, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:51.055213 28338 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:34:51.057148 28338 webserver.cc:460] Webserver started at http://127.15.198.130:44185/ using document root <none> and password file <none>
I20260806 22:34:51.058024 28338 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:51.058141 28338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:51.058490 28338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:51.062397 28338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "631bd9cd9b6b4bfcaa7dddc64339c074"
format_stamp: "Formatted at 2026-08-06 22:34:51 on dist-test-slave-h3ft"
server_key: "3174af179d455fdf1c07d1bbc4f8b83a"
server_key_iv: "2d40850f3ae47094c9e4dd002c4b3de3"
server_key_version: "encryptionkey@0"
I20260806 22:34:51.063449 28338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "631bd9cd9b6b4bfcaa7dddc64339c074"
format_stamp: "Formatted at 2026-08-06 22:34:51 on dist-test-slave-h3ft"
server_key: "3174af179d455fdf1c07d1bbc4f8b83a"
server_key_iv: "2d40850f3ae47094c9e4dd002c4b3de3"
server_key_version: "encryptionkey@0"
I20260806 22:34:51.070128 28338 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 22:34:51.074780 28363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:51.076625 28338 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:34:51.076839 28338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "631bd9cd9b6b4bfcaa7dddc64339c074"
format_stamp: "Formatted at 2026-08-06 22:34:51 on dist-test-slave-h3ft"
server_key: "3174af179d455fdf1c07d1bbc4f8b83a"
server_key_iv: "2d40850f3ae47094c9e4dd002c4b3de3"
server_key_version: "encryptionkey@0"
I20260806 22:34:51.077119 28338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:51.105882 28338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:51.111958 28338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:51.112351 28338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:51.113775 28338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:51.115695 28338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:51.115788 28338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:51.115973 28338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:51.116035 28338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:51.156044 28338 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:35125
I20260806 22:34:51.156100 28528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:35125 every 8 connection(s)
I20260806 22:34:51.157899 28338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:34:51.162684 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 28338
I20260806 22:34:51.162933 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:34:51.163501 16154 external_mini_cluster.cc:1428] Setting key 1b5e853db76f75f5362dfb91eed29210
Aug 06 22:34:51 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055691, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:34:51.188179 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:51.161457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:44563 (local address 127.15.198.190:34327)
0806 22:34:51.162043 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:51.162050 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:51.163817 (+ 1767us) server_negotiation.cc:407] Connection header received
0806 22:34:51.166783 (+ 2966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:51.166789 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:51.166901 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:51.167083 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:51.169360 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:51.170663 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:51.172773 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:51.173604 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:51.173781 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:51.178435 (+ 4654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:51.178445 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:51.178453 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:51.178511 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:51.182550 (+ 4039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:51.183912 (+ 1362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:51.183920 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:51.183926 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:51.184150 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:51.184874 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:51.184879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:51.184882 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:51.185288 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:51.185484 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:51.186825 (+ 1341us) server_negotiation.cc:299] Negotiation successful
0806 22:34:51.187383 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":323,"threads_started":1}
I20260806 22:34:51.189502 28532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:51.162387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:44563)
0806 22:34:51.163559 (+ 1172us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:51.163626 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:51.166372 (+ 2746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:51.167422 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:51.167434 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:51.168197 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:51.169119 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:51.169147 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:51.170922 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:51.170927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:51.172556 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:51.172593 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:51.173955 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:51.173963 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:51.174244 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:51.175281 (+ 1037us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:51.175311 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:51.178074 (+ 2763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:51.182732 (+ 4658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:51.182746 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:51.182796 (+ 50us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:51.183691 (+ 895us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:51.184313 (+ 622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:51.184321 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:51.184328 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:51.184699 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:51.185458 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:51.185469 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:51.186135 (+ 666us) client_negotiation.cc:769] Sending connection context
0806 22:34:51.186585 (+ 450us) client_negotiation.cc:240] Negotiation successful
0806 22:34:51.187452 (+ 867us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"thread_start_us":463,"threads_started":1}
I20260806 22:34:51.198127 28529 heartbeater.cc:347] Connected to a master server at 127.15.198.190:34327
I20260806 22:34:51.198740 28529 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:51.200075 28529 heartbeater.cc:511] Master 127.15.198.190:34327 requested a full tablet report, sending...
I20260806 22:34:51.202481 28063 ts_manager.cc:194] Registered new tserver with Master: 631bd9cd9b6b4bfcaa7dddc64339c074 (127.15.198.130:35125)
I20260806 22:34:51.203866 28063 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:44563
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
I20260806 22:34:51.252604 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:34327
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:41979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:34:51.512092 28331 heartbeater.cc:503] Master 127.15.198.190:34327 was elected leader, sending a full tablet report...
W20260806 22:34:51.685606 28535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:51.686106 28535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:51.686335 28535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:51.695501 28535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:51.695701 28535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:51.695909 28535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:34:51.707747 28535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:41979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:34327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:51.709808 28535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:51.711607 28535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:51.729727 28548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:51.729647 28542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:51.729770 28544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:51.731560 28535 server_base.cc:1034] running on GCE node
I20260806 22:34:51.732532 28535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:51.733946 28535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:51.735284 28535 hybrid_clock.cc:648] HybridClock initialized: now 1786055691735228 us; error 43 us; skew 500 ppm
I20260806 22:34:51.735765 28535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:51 dist-test-slave-h3ft krb5kdc[27996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055691, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:51.740078 28535 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:34:51.742174 28535 webserver.cc:460] Webserver started at http://127.15.198.131:42957/ using document root <none> and password file <none>
I20260806 22:34:51.743234 28535 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:51.743417 28535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:51.743840 28535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:34:51.748097 28535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "14b859fb2b35407bb414512e699955a0"
format_stamp: "Formatted at 2026-08-06 22:34:51 on dist-test-slave-h3ft"
server_key: "799892d5d4ac1e2240491bfc4a0d079e"
server_key_iv: "9d3ad2e5961079b6ba9b64e84b044cc3"
server_key_version: "encryptionkey@0"
I20260806 22:34:51.749186 28535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "14b859fb2b35407bb414512e699955a0"
format_stamp: "Formatted at 2026-08-06 22:34:51 on dist-test-slave-h3ft"
server_key: "799892d5d4ac1e2240491bfc4a0d079e"
server_key_iv: "9d3ad2e5961079b6ba9b64e84b044cc3"
server_key_version: "encryptionkey@0"
I20260806 22:34:51.756650 28535 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260806 22:34:51.761891 28558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:51.763844 28535 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:34:51.764110 28535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "14b859fb2b35407bb414512e699955a0"
format_stamp: "Formatted at 2026-08-06 22:34:51 on dist-test-slave-h3ft"
server_key: "799892d5d4ac1e2240491bfc4a0d079e"
server_key_iv: "9d3ad2e5961079b6ba9b64e84b044cc3"
server_key_version: "encryptionkey@0"
I20260806 22:34:51.764410 28535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:51.788605 28535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:51.794687 28535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:51.795220 28535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:51.796571 28535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:51.798594 28535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:51.798727 28535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:51.798923 28535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:51.799047 28535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:51.841847 28535 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:44485
I20260806 22:34:51.841890 28716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:44485 every 8 connection(s)
I20260806 22:34:51.843638 28535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:34:51.853219 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 28535
I20260806 22:34:51.853498 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:34:51.854054 16154 external_mini_cluster.cc:1428] Setting key 53b2b8fffe8634086a6331d660272db4
Aug 06 22:34:51 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055691, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:34:51.870913 28720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:51.847610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:53575 (local address 127.15.198.190:34327)
0806 22:34:51.848452 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:51.848458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:51.849653 (+ 1195us) server_negotiation.cc:407] Connection header received
0806 22:34:51.852154 (+ 2501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:51.852162 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:51.852297 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:51.852449 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:51.854880 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:51.856102 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:51.858073 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:51.858621 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:51.858734 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:51.863491 (+ 4757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:51.863498 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:51.863501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:51.863548 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:51.866704 (+ 3156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:51.867674 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:51.867679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:51.867682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:51.867833 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:51.868332 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:51.868340 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:51.868342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:51.868809 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:51.869067 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:51.870138 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0806 22:34:51.870471 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":505,"threads_started":1}
I20260806 22:34:51.872233 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:51.848506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:53575)
0806 22:34:51.849393 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:51.849469 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:51.851749 (+ 2280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:51.852864 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:51.852876 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:51.853711 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:51.854618 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:51.854646 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:51.856310 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:51.856314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:51.857857 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:51.857874 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:51.858908 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:51.858914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:51.859299 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:51.860206 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:51.860255 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:51.863153 (+ 2898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:34:51.866867 (+ 3714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:51.866875 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:51.866894 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:51.867519 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:51.867960 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:34:51.867964 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:34:51.867966 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:34:51.868188 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:34:51.869040 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:34:51.869049 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:34:51.869563 (+ 514us) client_negotiation.cc:769] Sending connection context
0806 22:34:51.869982 (+ 419us) client_negotiation.cc:240] Negotiation successful
0806 22:34:51.870720 (+ 738us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":334,"threads_started":1}
I20260806 22:34:51.873968 28717 heartbeater.cc:347] Connected to a master server at 127.15.198.190:34327
I20260806 22:34:51.874359 28717 heartbeater.cc:464] Registering TS with master...
I20260806 22:34:51.875361 28717 heartbeater.cc:511] Master 127.15.198.190:34327 requested a full tablet report, sending...
I20260806 22:34:51.877140 28063 ts_manager.cc:194] Registered new tserver with Master: 14b859fb2b35407bb414512e699955a0 (127.15.198.131:44485)
I20260806 22:34:51.878064 28063 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:53575
I20260806 22:34:51.884210 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.129.keytab.
Entry for principal oryx/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.129.keytab.
WARNING: no policy specified for oryx/127.15.198.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.198.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.130.keytab.
Entry for principal oryx/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.130.keytab.
WARNING: no policy specified for oryx/127.15.198.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.198.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.131.keytab.
Entry for principal oryx/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.131.keytab.
I20260806 22:34:52.115414 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 28150
Aug 06 22:34:52 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:34:52.137970 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:52.117117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52512 (local address 127.15.198.129:44831)
0806 22:34:52.117995 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:52.118004 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:52.118040 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:34:52.118123 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:52.118132 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:52.118417 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:52.118589 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:52.120393 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:52.121682 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:52.123422 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:52.124169 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:52.124354 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:52.130087 (+ 5733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:52.130110 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:52.130133 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:52.130192 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:52.134006 (+ 3814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:52.135094 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:52.135105 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:52.135108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:52.135316 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:52.135911 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:52.135915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:52.135918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:52.136388 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:52.136819 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:52.137135 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 22:34:52.137490 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":442,"threads_started":1}
I20260806 22:34:52.139863 28326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:52.208079 28529 heartbeater.cc:503] Master 127.15.198.190:34327 was elected leader, sending a full tablet report...
I20260806 22:34:52.235025 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 28150
I20260806 22:34:52.268383 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 28338
Aug 06 22:34:52 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.130@KRBTEST.COM
I20260806 22:34:52.289626 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:52.269886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33928 (local address 127.15.198.130:35125)
0806 22:34:52.270757 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:52.270766 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:52.270802 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:34:52.270885 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:52.270894 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:52.271289 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:52.271472 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:52.273004 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:52.274137 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:52.275791 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:52.276464 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:52.276576 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:52.281795 (+ 5219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:52.281817 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:52.281840 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:52.281974 (+ 134us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:52.285974 (+ 4000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:52.286896 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:52.286906 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:52.286908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:52.287181 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:52.287605 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:52.287609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:52.287611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:52.288086 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:52.288363 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:52.288818 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 22:34:52.289131 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":465,"threads_started":1}
I20260806 22:34:52.291440 28521 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:52.386145 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 28338
I20260806 22:34:52.411865 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 28535
Aug 06 22:34:52 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.131@KRBTEST.COM
I20260806 22:34:52.433516 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:52.413239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60414 (local address 127.15.198.131:44485)
0806 22:34:52.413992 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:52.414001 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:52.414035 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:34:52.414120 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:52.414128 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:52.414482 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:52.414648 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:52.416391 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:52.417702 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:52.419320 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:52.420012 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:52.420140 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:52.425966 (+ 5826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:52.425988 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:52.426016 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:52.426080 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:52.429788 (+ 3708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:52.430720 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:52.430730 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:52.430733 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:52.430912 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:52.431607 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:52.431612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:52.431615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:52.432101 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:52.432423 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:52.432759 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 22:34:52.433030 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":423,"threads_started":1}
I20260806 22:34:52.435452 28708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:52.538058 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 28535
I20260806 22:34:52.570382 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 28015
I20260806 22:34:52.572348 28104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:34:52.696162 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 28015
I20260806 22:34:52.731259 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:34327
--webserver_interface=127.15.198.190
--webserver_port=43233
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:41979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:34327
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:34:53.166543 28742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:53.167088 28742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:53.167275 28742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:53.175735 28742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:34:53.175971 28742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:53.176133 28742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:53.176250 28742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:34:53.176333 28742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:34:53.188845 28742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:41979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:34327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:34327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=43233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:53.190737 28742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:53.192533 28742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:53.207291 28749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:53.207180 28751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:53.207223 28755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:53.208478 28742 server_base.cc:1034] running on GCE node
I20260806 22:34:53.209602 28742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:53.211654 28742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:53.212946 28742 hybrid_clock.cc:648] HybridClock initialized: now 1786055693212886 us; error 64 us; skew 500 ppm
I20260806 22:34:53.213390 28742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:53 dist-test-slave-h3ft krb5kdc[27996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055693, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:53.217257 28742 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:34:53.219159 28742 webserver.cc:460] Webserver started at http://127.15.198.190:43233/ using document root <none> and password file <none>
I20260806 22:34:53.220142 28742 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:53.220274 28742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:53.227532 28742 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 22:34:53.231246 28763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:53.233086 28742 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:34:53.233286 28742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "055e5a87859541429b98661d03745f94"
format_stamp: "Formatted at 2026-08-06 22:34:49 on dist-test-slave-h3ft"
server_key: "8f0d80f6d9fbbf1b591e9ab6ec108bc9"
server_key_iv: "ffc528b74ec668dcd70a4361088cb7db"
server_key_version: "encryptionkey@0"
I20260806 22:34:53.234508 28742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:53.265301 28742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:53.270314 28742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:53.270612 28742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:53.294543 28742 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:34327
I20260806 22:34:53.294559 28836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:34327 every 8 connection(s)
I20260806 22:34:53.296733 28742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:34:53.302354 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 28742
I20260806 22:34:53.303637 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:44831
--local_ip_for_outbound_sockets=127.15.198.129
--tserver_master_addrs=127.15.198.190:34327
--webserver_port=38249
--webserver_interface=127.15.198.129
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:41979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.198.129
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:34:53.309207 28837 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:34:53.315779 28837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94: Bootstrap starting.
I20260806 22:34:53.342024 28837 log.cc:824] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94: Log is configured to *not* fsync() on all Append() calls
I20260806 22:34:53.354133 28837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:34:53.354898 28837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94: Bootstrap complete.
I20260806 22:34:53.365984 28837 raft_consensus.cc:348] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e5a87859541429b98661d03745f94" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 34327 } }
I20260806 22:34:53.367489 28837 raft_consensus.cc:729] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 055e5a87859541429b98661d03745f94, State: Initialized, Role: FOLLOWER
I20260806 22:34:53.368470 28837 consensus_queue.cc:260] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e5a87859541429b98661d03745f94" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 34327 } }
I20260806 22:34:53.368788 28837 raft_consensus.cc:388] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:34:53.368949 28837 raft_consensus.cc:482] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:34:53.369190 28837 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:34:53.374089 28837 raft_consensus.cc:504] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e5a87859541429b98661d03745f94" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 34327 } }
I20260806 22:34:53.374817 28837 leader_election.cc:302] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 055e5a87859541429b98661d03745f94; no voters:
I20260806 22:34:53.375691 28837 leader_election.cc:288] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:34:53.375859 28844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:34:53.378723 28837 sys_catalog.cc:564] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:34:53.380287 28844 raft_consensus.cc:686] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [term 2 LEADER]: Becoming Leader. State: Replica: 055e5a87859541429b98661d03745f94, State: Running, Role: LEADER
I20260806 22:34:53.381227 28844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e5a87859541429b98661d03745f94" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 34327 } }
I20260806 22:34:53.386092 28845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "055e5a87859541429b98661d03745f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e5a87859541429b98661d03745f94" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 34327 } } }
I20260806 22:34:53.386176 28846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 055e5a87859541429b98661d03745f94. Latest consensus state: current_term: 2 leader_uuid: "055e5a87859541429b98661d03745f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e5a87859541429b98661d03745f94" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 34327 } } }
I20260806 22:34:53.386586 28845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:53.386653 28846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94 [sys.catalog]: This master's current role is: LEADER
I20260806 22:34:53.397565 28860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:34:53.405827 28860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:34:53.407981 28860 catalog_manager.cc:1259] Loaded cluster ID: b0c321be95e84cf2920f1061b540ead1
I20260806 22:34:53.408147 28860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:34:53.413084 28860 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:34:53.415299 28860 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 055e5a87859541429b98661d03745f94: Loaded TSK: 0
I20260806 22:34:53.417727 28860 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:34:53.767768 28840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:53.768172 28840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:53.768357 28840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:53.777143 28840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:53.777292 28840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:53.777508 28840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:34:53.789820 28840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:41979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.129.keytab
--principal=oryx/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:44831
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=38249
--tserver_master_addrs=127.15.198.190:34327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:53.791865 28840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:53.793689 28840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:53.810173 28878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:53.810230 28881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:53.810782 28876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:53.811427 28840 server_base.cc:1034] running on GCE node
I20260806 22:34:53.812227 28840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:53.813454 28840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:53.814694 28840 hybrid_clock.cc:648] HybridClock initialized: now 1786055693814611 us; error 65 us; skew 500 ppm
I20260806 22:34:53.815176 28840 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:53 dist-test-slave-h3ft krb5kdc[27996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055693, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:53.819167 28840 init.cc:377] Logged in from keytab as oryx/127.15.198.129@KRBTEST.COM (short username oryx)
I20260806 22:34:53.821053 28840 webserver.cc:460] Webserver started at http://127.15.198.129:38249/ using document root <none> and password file <none>
I20260806 22:34:53.821938 28840 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:53.822074 28840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:53.828882 28840 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:34:53.832372 28894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:53.834252 28840 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:34:53.834503 28840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "3ddc06c44ab34fa59e3cecd7a24dc2c9"
format_stamp: "Formatted at 2026-08-06 22:34:50 on dist-test-slave-h3ft"
server_key: "be36becff6a573a978b5bb1aad0f10c8"
server_key_iv: "d98f6609f8491505225fe182cac8c7e1"
server_key_version: "encryptionkey@0"
I20260806 22:34:53.835883 28840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:53.862427 28840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:53.868162 28840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:53.868564 28840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:53.869951 28840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:53.871815 28840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:53.871968 28840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:53.872143 28840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:53.872248 28840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:53.933795 28840 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:44831
I20260806 22:34:53.933851 29064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:44831 every 8 connection(s)
I20260806 22:34:53.935873 28840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:34:53.939373 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 28840
I20260806 22:34:53.940670 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:35125
--local_ip_for_outbound_sockets=127.15.198.130
--tserver_master_addrs=127.15.198.190:34327
--webserver_port=44185
--webserver_interface=127.15.198.130
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:41979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.198.130
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:34:53 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:53 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:53.965978 29068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:53.940685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:52111)
0806 22:34:53.942744 (+ 2059us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:53.942850 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:53.947196 (+ 4346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:53.949633 (+ 2437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:53.949647 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:53.950834 (+ 1187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:53.952100 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:53.952135 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:53.954391 (+ 2256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:53.954400 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:53.957001 (+ 2601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:53.957024 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:53.958440 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:53.958451 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:53.958759 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:53.959766 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:53.959795 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:53.964552 (+ 4757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1423,"thread_start_us":1112,"threads_started":1}
W20260806 22:34:53.966460 29067 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:53.940515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:52111 (local address 127.15.198.190:34327)
0806 22:34:53.941770 (+ 1255us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:53.941797 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:53.943167 (+ 1370us) server_negotiation.cc:407] Connection header received
0806 22:34:53.948332 (+ 5165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:53.948347 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:53.948932 (+ 585us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:53.949329 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:53.952399 (+ 3070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:53.954124 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:53.957218 (+ 3094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:53.958084 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:53.958279 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:53.965172 (+ 6893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:52111: BlockingRecv error: recv got EOF from 127.15.198.129:52111 (error 108)
Metrics: {"server-negotiator.queue_time_us":686,"spinlock_wait_cycles":2816,"thread_start_us":514,"threads_started":1}
W20260806 22:34:53.966874 29065 heartbeater.cc:644] Failed to heartbeat to 127.15.198.190:34327 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.198.190:34327: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Aug 06 22:34:53 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:53 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:53.977312 29068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:53.967924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:50777)
0806 22:34:53.968157 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:53.968177 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:53.968335 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:53.968733 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:53.968738 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:53.969183 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:53.969357 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:53.969367 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:53.970483 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:53.970496 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:53.971508 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:53.971522 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:53.972277 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:53.972283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:53.972475 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:53.973315 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:53.973337 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:53.976957 (+ 3620us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 22:34:53.977708 29067 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:53.968173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:50777 (local address 127.15.198.190:34327)
0806 22:34:53.968388 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:53.968397 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:53.968421 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:34:53.968501 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:53.968505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:53.968583 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:53.968698 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:53.969541 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:53.970318 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:53.971656 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:53.972116 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:53.972263 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:53.977214 (+ 4951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:50777: BlockingRecv error: recv got EOF from 127.15.198.129:50777 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 22:34:53 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:53 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:53.989372 29068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:53.978173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:46965)
0806 22:34:53.978438 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:53.978466 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:53.978606 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:53.979036 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:53.979044 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:53.979581 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:53.979880 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:53.979899 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:53.981118 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:53.981127 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:53.982956 (+ 1829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:53.982979 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:53.983906 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:53.983913 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:53.984136 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:53.984992 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:53.985019 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:53.988998 (+ 3979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":77}
W20260806 22:34:53.989859 29065 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:34:53.990177 29067 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:53.978433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:46965 (local address 127.15.198.190:34327)
0806 22:34:53.978638 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:53.978642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:53.978660 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:34:53.978720 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:53.978761 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:53.978851 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:53.978936 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:53.980070 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:53.980938 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:53.983153 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:53.983734 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:53.983843 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:53.989357 (+ 5514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:46965: BlockingRecv error: recv got EOF from 127.15.198.129:46965 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 22:34:54.376664 29072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:54.377074 29072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:54.377255 29072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:54.387115 29072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:54.387282 29072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:54.387494 29072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:34:54.399097 29072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:41979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.130.keytab
--principal=oryx/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:35125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=44185
--tserver_master_addrs=127.15.198.190:34327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:54.401057 29072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:54.402912 29072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:54.419365 29080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:54.419353 29078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:54.419492 29077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:34:54.421150 29072 server_base.cc:1034] running on GCE node
I20260806 22:34:54.421983 29072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:54.423277 29072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:54.424528 29072 hybrid_clock.cc:648] HybridClock initialized: now 1786055694424473 us; error 67 us; skew 500 ppm
I20260806 22:34:54.424947 29072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:54 dist-test-slave-h3ft krb5kdc[27996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055694, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:54.428881 29072 init.cc:377] Logged in from keytab as oryx/127.15.198.130@KRBTEST.COM (short username oryx)
I20260806 22:34:54.430686 29072 webserver.cc:460] Webserver started at http://127.15.198.130:44185/ using document root <none> and password file <none>
I20260806 22:34:54.431689 29072 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:54.431881 29072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:54.438778 29072 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 22:34:54.442415 29090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:54.444212 29072 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:34:54.444444 29072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "631bd9cd9b6b4bfcaa7dddc64339c074"
format_stamp: "Formatted at 2026-08-06 22:34:51 on dist-test-slave-h3ft"
server_key: "3174af179d455fdf1c07d1bbc4f8b83a"
server_key_iv: "2d40850f3ae47094c9e4dd002c4b3de3"
server_key_version: "encryptionkey@0"
I20260806 22:34:54.445655 29072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:54.468787 29072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:54.474237 29072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:54.474614 29072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:54.475955 29072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:54.477698 29072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:54.477847 29072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:54.478008 29072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:54.478107 29072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:54.518414 29072 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:35125
I20260806 22:34:54.518489 29261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:35125 every 8 connection(s)
I20260806 22:34:54.520408 29072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:34:54.527918 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 29072
I20260806 22:34:54.528841 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:44485
--local_ip_for_outbound_sockets=127.15.198.131
--tserver_master_addrs=127.15.198.190:34327
--webserver_port=42957
--webserver_interface=127.15.198.131
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:41979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.198.131
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:34:54 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:54 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:34:54.543467 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:54.523647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:53325 (local address 127.15.198.190:34327)
0806 22:34:54.524276 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:54.524281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:54.525247 (+ 966us) server_negotiation.cc:407] Connection header received
0806 22:34:54.527542 (+ 2295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:54.527547 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:54.527632 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:54.527742 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:54.530413 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:54.531478 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:54.534314 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:54.534795 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:54.534942 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:54.543158 (+ 8216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:53325: BlockingRecv error: recv got EOF from 127.15.198.130:53325 (error 108)
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":335,"threads_started":1}
I20260806 22:34:54.544580 29266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:54.524211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:53325)
0806 22:34:54.525044 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:54.525099 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:54.527166 (+ 2067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:54.528105 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:54.528119 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:54.529094 (+ 975us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:54.530125 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:54.530161 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:54.531692 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:54.531700 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:54.534097 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:54.534118 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:54.535132 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:54.535144 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:54.535543 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:54.536543 (+ 1000us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:54.536590 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:54.542756 (+ 6166us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":328,"threads_started":1}
W20260806 22:34:54.545504 29262 heartbeater.cc:644] Failed to heartbeat to 127.15.198.190:34327 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.198.190:34327: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Aug 06 22:34:54 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:54 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:54.556591 29266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:54.546374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:42293)
0806 22:34:54.546613 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:54.546638 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:54.546755 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:54.547182 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:54.547186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:54.547587 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:54.547769 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:54.547778 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:54.549333 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:54.549337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:54.550358 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:54.550372 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:54.551551 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:54.551557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:54.551699 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:54.552424 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:54.552444 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:54.556176 (+ 3732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":71}
W20260806 22:34:54.556638 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:54.546508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:42293 (local address 127.15.198.190:34327)
0806 22:34:54.546738 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:54.546746 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:54.546776 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:34:54.546862 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:54.546870 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:54.546975 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:54.547134 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:54.547953 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:54.549178 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:54.550547 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:54.551400 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:54.551551 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:54.556416 (+ 4865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:42293: BlockingRecv error: recv got EOF from 127.15.198.130:42293 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 22:34:54 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:54 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:54.566607 29266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:54.557496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:52973)
0806 22:34:54.557705 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:54.557735 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:54.557865 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:54.558301 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:54.558306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:54.558690 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:54.558870 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:54.558880 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:54.560027 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:54.560035 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:54.561146 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:54.561159 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:54.561933 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:54.561940 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:54.562160 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:54.562910 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:54.562929 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:54.566229 (+ 3300us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 22:34:54.566831 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:54.557751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:52973 (local address 127.15.198.190:34327)
0806 22:34:54.558001 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:54.558006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:54.558021 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:34:54.558082 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:54.558086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:54.558155 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:54.558272 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:54.559066 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:54.559871 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:54.561298 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:54.561776 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:54.561921 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:54.566500 (+ 4579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:52973: BlockingRecv error: recv got EOF from 127.15.198.130:52973 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 22:34:54.567210 29262 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:34:54.960807 29270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:34:54.961287 29270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:34:54.961499 29270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:34:54.969914 29270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:34:54.970142 29270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:34:54.970362 29270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:34:54.982537 29270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:41979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/krb5kdc/oryx_127.15.198.131.keytab
--principal=oryx/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:44485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=42957
--tserver_master_addrs=127.15.198.190:34327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:34:54.984608 29270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:34:54.986383 29270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:34:55.004969 29277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:55.004980 29278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:34:55.005261 29283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
Aug 06 22:34:55 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:55 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:55.008441 29270 server_base.cc:1034] running on GCE node
I20260806 22:34:55.009527 29270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:34:55.010141 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:54.990849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:58253)
0806 22:34:54.991619 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:54.991654 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:54.991894 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:54.992450 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:54.992455 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:54.993089 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:54.993343 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:54.993357 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:54.994657 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:54.994666 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:54.996566 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:54.996590 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:54.998159 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:54.998166 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:54.998401 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:55.005356 (+ 6955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:55.005405 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:55.009747 (+ 4342us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":413,"threads_started":1}
W20260806 22:34:55.010435 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:54.991072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:58253 (local address 127.15.198.190:34327)
0806 22:34:54.991388 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:54.991396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:54.991814 (+ 418us) server_negotiation.cc:407] Connection header received
0806 22:34:54.992121 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:54.992127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:54.992222 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:54.992357 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:54.993572 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:54.994475 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:54.996797 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:54.997642 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:54.997763 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:55.010121 (+ 12358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:58253: BlockingRecv error: recv got EOF from 127.15.198.129:58253 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
I20260806 22:34:55.011154 29270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:34:55.012408 29270 hybrid_clock.cc:648] HybridClock initialized: now 1786055695012308 us; error 80 us; skew 500 ppm
I20260806 22:34:55.012837 29270 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:34:55 dist-test-slave-h3ft krb5kdc[27996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055695, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:34:55.017113 29270 init.cc:377] Logged in from keytab as oryx/127.15.198.131@KRBTEST.COM (short username oryx)
I20260806 22:34:55.019189 29270 webserver.cc:460] Webserver started at http://127.15.198.131:42957/ using document root <none> and password file <none>
I20260806 22:34:55.020125 29270 fs_manager.cc:359] Metadata directory not provided
I20260806 22:34:55.020342 29270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:34:55.027361 29270 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 22:34:55.030911 29293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:55.032987 29270 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:34:55.033185 29270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "14b859fb2b35407bb414512e699955a0"
format_stamp: "Formatted at 2026-08-06 22:34:51 on dist-test-slave-h3ft"
server_key: "799892d5d4ac1e2240491bfc4a0d079e"
server_key_iv: "9d3ad2e5961079b6ba9b64e84b044cc3"
server_key_version: "encryptionkey@0"
I20260806 22:34:55.034416 29270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:34:55.050140 29270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:34:55.056339 29270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:34:55.056685 29270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:34:55.057967 29270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:34:55.059965 29270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:34:55.060081 29270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:55.060185 29270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:34:55.060252 29270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:34:55.104216 29270 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:44485
I20260806 22:34:55.104380 29451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:44485 every 8 connection(s)
I20260806 22:34:55.106155 29270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:34:55.114532 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 29270
Aug 06 22:34:55 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:55 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:34:55.128425 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:55.109869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:58629 (local address 127.15.198.190:34327)
0806 22:34:55.110158 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:55.110167 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:55.111698 (+ 1531us) server_negotiation.cc:407] Connection header received
0806 22:34:55.114050 (+ 2352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:55.114059 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:55.114192 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:55.114371 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:55.117126 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.118000 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.119973 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.120748 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.120923 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:55.127852 (+ 6929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.131:58629: BlockingRecv error: recv got EOF from 127.15.198.131:58629 (error 108)
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":99}
I20260806 22:34:55.129598 29458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:55.110455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:58629)
0806 22:34:55.111466 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:55.111516 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:55.113610 (+ 2094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:55.114717 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:55.114729 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:55.115643 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:55.116869 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:55.116899 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.118218 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.118223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:55.119802 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:55.119818 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.121076 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.121085 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:55.121397 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:55.122351 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:55.122389 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:55.127609 (+ 5220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":435,"threads_started":1}
W20260806 22:34:55.130641 29452 heartbeater.cc:644] Failed to heartbeat to 127.15.198.190:34327 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.198.190:34327: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
I20260806 22:34:55.136184 29463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:55.115936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56464 (local address 127.15.198.190:34327)
0806 22:34:55.117047 (+ 1111us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:55.117052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:55.117068 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:34:55.117121 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:55.117125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:55.117189 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:55.117327 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:55.119280 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.120576 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.122040 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.122723 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.122872 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:55.126252 (+ 3380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:34:55.126267 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:34:55.126307 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:34:55.126374 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:34:55.131067 (+ 4693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:55.132165 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:55.132173 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:55.132183 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:55.132677 (+ 494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:34:55.133468 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:34:55.133475 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:34:55.133478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:34:55.134318 (+ 840us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:34:55.134723 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0806 22:34:55.135304 (+ 581us) server_negotiation.cc:299] Negotiation successful
0806 22:34:55.135789 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":937,"thread_start_us":768,"threads_started":1}
Aug 06 22:34:55 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:55 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:55.142336 29458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:55.131582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:34779)
0806 22:34:55.131810 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:55.131842 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:55.131973 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:55.132410 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:55.132414 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:55.132951 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:55.133320 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:55.133335 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.134783 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.134788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:55.136138 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:55.136154 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.137081 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.137087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:55.137278 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:55.137997 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:55.138018 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:55.142074 (+ 4056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":70}
W20260806 22:34:55.142946 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:55.131691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:34779 (local address 127.15.198.190:34327)
0806 22:34:55.131975 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:55.131982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:55.132002 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:34:55.132093 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:55.132100 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:55.132186 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:55.132302 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:55.133491 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.134613 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.136297 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.136910 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.137065 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:55.142317 (+ 5252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.131:34779: BlockingRecv error: recv got EOF from 127.15.198.131:34779 (error 108)
Metrics: {"server-negotiator.queue_time_us":155}
Aug 06 22:34:55 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:55 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:55.153266 29458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:55.143413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:39087)
0806 22:34:55.143617 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:55.143648 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:55.143791 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:55.144224 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:55.144228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:55.144588 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:55.144780 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:55.144790 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.145972 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.145976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:55.147070 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:55.147097 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.147923 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.147927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:55.148093 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:55.148831 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:55.148851 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:55.152802 (+ 3951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 22:34:55.153488 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:55.143369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:39087 (local address 127.15.198.190:34327)
0806 22:34:55.143580 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:55.143585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:55.143760 (+ 175us) server_negotiation.cc:407] Connection header received
0806 22:34:55.143950 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:55.143955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:55.144036 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:55.144150 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:55.144960 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.145813 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.147240 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.147761 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.147880 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:55.153150 (+ 5270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.131:39087: BlockingRecv error: recv got EOF from 127.15.198.131:39087 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 22:34:55.153849 29452 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 22:34:55 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:55 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:55.578593 29465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:55.568075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:46137)
0806 22:34:55.568971 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:55.569025 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:55.569195 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:55.569583 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:55.569606 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:55.570383 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:55.570633 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:55.570644 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.571869 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.571875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:55.572914 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:55.572927 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.573776 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.573784 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:55.573990 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:55.574714 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:55.574736 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:55.578296 (+ 3560us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":747,"thread_start_us":589,"threads_started":1}
W20260806 22:34:55.578889 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:55.568219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:46137 (local address 127.15.198.190:34327)
0806 22:34:55.568560 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:55.568571 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:55.569140 (+ 569us) server_negotiation.cc:407] Connection header received
0806 22:34:55.569329 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:55.569334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:55.569413 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:55.569510 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:55.570822 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.571711 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.573083 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:55.573576 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:55.573734 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:55.578579 (+ 4845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:46137: BlockingRecv error: recv got EOF from 127.15.198.130:46137 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 06 22:34:56 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:56 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:56.022632 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:56.011781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:43881)
0806 22:34:56.012460 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:56.012484 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:56.012711 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:56.013135 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:56.013140 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:56.013757 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:56.013978 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:56.013989 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:56.015283 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:56.015289 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:56.016278 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:56.016290 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:56.017130 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:56.017136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:56.017335 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:56.018209 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:56.018230 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:56.022270 (+ 4040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":386,"threads_started":1}
W20260806 22:34:56.022888 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:56.011852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:43881 (local address 127.15.198.190:34327)
0806 22:34:56.012136 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:56.012144 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:56.012618 (+ 474us) server_negotiation.cc:407] Connection header received
0806 22:34:56.012874 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:56.012879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:56.012961 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:56.013101 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:56.014164 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:56.015113 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:56.016444 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:56.016930 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:56.017089 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:56.022570 (+ 5481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:43881: BlockingRecv error: recv got EOF from 127.15.198.129:43881 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 22:34:56 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:56 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:56.165566 29467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:56.154629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:60577)
0806 22:34:56.155359 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:56.155380 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:56.155562 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:56.155939 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:56.155943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:56.156585 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:56.156835 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:56.156847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:56.157998 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:56.158004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:56.159216 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:56.159232 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:56.160033 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:56.160039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:56.160197 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:56.161001 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:56.161028 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:56.165176 (+ 4148us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":423,"threads_started":1}
W20260806 22:34:56.165712 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:56.154725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:60577 (local address 127.15.198.190:34327)
0806 22:34:56.155032 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:56.155043 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:56.155503 (+ 460us) server_negotiation.cc:407] Connection header received
0806 22:34:56.155688 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:56.155692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:56.155767 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:56.155857 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:56.156986 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:56.157813 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:56.159359 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:56.159823 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:56.159925 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:56.165434 (+ 5509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.131:60577: BlockingRecv error: recv got EOF from 127.15.198.131:60577 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 06 22:34:56 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:56 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:56.591755 29469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:56.580055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:43867)
0806 22:34:56.580868 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:56.580889 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:56.581125 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:56.581572 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:56.581578 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:56.582134 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:56.582421 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:56.582434 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:56.584221 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:56.584229 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:56.585813 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:56.585837 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:56.586699 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:56.586706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:56.586901 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:56.587718 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:56.587744 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:56.591438 (+ 3694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":519,"threads_started":1}
W20260806 22:34:56.592356 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:56.580330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:43867 (local address 127.15.198.190:34327)
0806 22:34:56.580634 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:56.580643 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:56.581015 (+ 372us) server_negotiation.cc:407] Connection header received
0806 22:34:56.581304 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:56.581315 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:56.581402 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:56.581527 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:56.582624 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:56.583952 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:56.586015 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:56.586502 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:56.586612 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:56.591771 (+ 5159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:43867: BlockingRecv error: recv got EOF from 127.15.198.130:43867 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
Aug 06 22:34:57 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:57 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:57.034449 29470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:57.024038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:36685)
0806 22:34:57.024661 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:57.024686 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:57.024879 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:57.025293 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:57.025298 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:57.025783 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:57.026007 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:57.026018 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:57.027290 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:57.027295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:57.028329 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:57.028343 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:57.029168 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:57.029174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:57.029375 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:57.030193 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:57.030214 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:57.034006 (+ 3792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":359,"threads_started":1}
W20260806 22:34:57.034672 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:57.024139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:36685 (local address 127.15.198.190:34327)
0806 22:34:57.024385 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:57.024393 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:57.024787 (+ 394us) server_negotiation.cc:407] Connection header received
0806 22:34:57.025043 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:57.025049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:57.025133 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:57.025253 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:57.026184 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:57.027118 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:57.028490 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:57.028982 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:57.029139 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:57.034307 (+ 5168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:36685: BlockingRecv error: recv got EOF from 127.15.198.129:36685 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 22:34:57 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:57 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:57.177718 29471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:57.167300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:33029)
0806 22:34:57.168076 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:57.168096 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:57.168307 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:57.168738 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:57.168743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:57.169403 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:57.169620 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:57.169632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:57.170801 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:57.170809 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:57.171816 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:57.171828 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:57.172673 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:57.172680 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:57.172859 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:57.173627 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:57.173648 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:57.177381 (+ 3733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":415,"threads_started":1}
W20260806 22:34:57.178083 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:57.167338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:33029 (local address 127.15.198.190:34327)
0806 22:34:57.167643 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:57.167652 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:57.168205 (+ 553us) server_negotiation.cc:407] Connection header received
0806 22:34:57.168473 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:57.168482 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:57.168569 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:57.168701 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:57.169791 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:57.170639 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:57.171985 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:57.172469 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:57.172652 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:57.177722 (+ 5070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.131:33029: BlockingRecv error: recv got EOF from 127.15.198.131:33029 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 06 22:34:57 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:57 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:57.603247 29472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:57.592968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:57609)
0806 22:34:57.593659 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:57.593678 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:57.593825 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:57.594293 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:57.594296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:57.594756 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:57.594983 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:57.594995 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:57.596183 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:57.596188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:57.597389 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:57.597405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:57.598278 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:57.598282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:57.598412 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:57.599315 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:57.599336 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:57.602749 (+ 3413us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":390,"threads_started":1}
W20260806 22:34:57.603323 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:57.593049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:57609 (local address 127.15.198.190:34327)
0806 22:34:57.593308 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:57.593313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:57.593797 (+ 484us) server_negotiation.cc:407] Connection header received
0806 22:34:57.594035 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:57.594040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:57.594139 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:57.594237 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:57.595213 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:57.596045 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:57.597557 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:57.598123 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:57.598260 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:57.603052 (+ 4792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:57609: BlockingRecv error: recv got EOF from 127.15.198.130:57609 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 06 22:34:58 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:58 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:58.046401 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:58.035836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:55737)
0806 22:34:58.036588 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:58.036612 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:58.036772 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:58.037185 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:58.037190 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:58.037707 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:58.037908 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:58.037919 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:58.039292 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:58.039297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:58.040619 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:58.040647 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:58.041452 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:58.041457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:58.041612 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:58.042421 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:58.042442 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:58.045994 (+ 3552us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":396,"threads_started":1}
W20260806 22:34:58.046641 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:58.035926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:55737 (local address 127.15.198.190:34327)
0806 22:34:58.036147 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:58.036153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:58.036771 (+ 618us) server_negotiation.cc:407] Connection header received
0806 22:34:58.036900 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:58.036904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:58.036994 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:58.037088 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:58.038082 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:58.039096 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:58.040820 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:58.041310 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:58.041426 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:58.046250 (+ 4824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:55737: BlockingRecv error: recv got EOF from 127.15.198.129:55737 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 06 22:34:58 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:58 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:58.189347 29474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:58.179259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:53189)
0806 22:34:58.179940 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:58.179961 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:58.180227 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:58.180631 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:58.180636 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:58.181128 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:58.181362 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:58.181372 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:58.182556 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:58.182565 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:58.183689 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:58.183710 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:58.184473 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:58.184480 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:58.184683 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:58.185482 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:58.185502 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:58.188884 (+ 3382us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":386,"threads_started":1}
W20260806 22:34:58.189455 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:58.179313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:53189 (local address 127.15.198.190:34327)
0806 22:34:58.179601 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:58.179610 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:58.180072 (+ 462us) server_negotiation.cc:407] Connection header received
0806 22:34:58.180383 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:58.180389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:58.180479 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:58.180587 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:58.181526 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:58.182392 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:58.183859 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:58.184307 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:58.184454 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:58.189164 (+ 4710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.131:53189: BlockingRecv error: recv got EOF from 127.15.198.131:53189 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 06 22:34:58 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:58 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:58.615811 29475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:58.604790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:55321)
0806 22:34:58.605531 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:58.605558 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:58.605795 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:58.606349 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:58.606354 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:58.606971 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:58.607237 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:58.607250 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:58.608792 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:58.608797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:58.609889 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:58.609905 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:58.610710 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:58.610715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:58.610859 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:58.611744 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:58.611765 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:58.615432 (+ 3667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":393,"threads_started":1}
W20260806 22:34:58.616068 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:58.604930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:55321 (local address 127.15.198.190:34327)
0806 22:34:58.605255 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:58.605264 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:58.605708 (+ 444us) server_negotiation.cc:407] Connection header received
0806 22:34:58.606024 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:58.606033 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:58.606157 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:58.606255 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:58.607418 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:58.608637 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:58.610058 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:58.610557 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:58.610673 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:58.615782 (+ 5109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:55321: BlockingRecv error: recv got EOF from 127.15.198.130:55321 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
Aug 06 22:34:59 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:59 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:59.059959 29477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:59.047910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:37659)
0806 22:34:59.048797 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:59.048831 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:59.049007 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:59.049468 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:59.049473 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:59.050031 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:59.050407 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:59.050423 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:59.051980 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:59.051985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:59.053379 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:59.053398 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:59.054332 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:59.054339 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:59.054580 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:59.055547 (+ 967us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:59.055576 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:59.059601 (+ 4025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":472,"threads_started":1}
W20260806 22:34:59.060271 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:59.048196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:37659 (local address 127.15.198.190:34327)
0806 22:34:59.048456 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:59.048462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:59.049016 (+ 554us) server_negotiation.cc:407] Connection header received
0806 22:34:59.049191 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:59.049195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:59.049281 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:59.049387 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:59.050641 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:59.051793 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:59.053583 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:59.054120 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:59.054295 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:59.059967 (+ 5672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:37659: BlockingRecv error: recv got EOF from 127.15.198.129:37659 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 22:34:59 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:59 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:59.201531 29478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:59.190683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:59289)
0806 22:34:59.191472 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:59.191491 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:59.191647 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:59.192026 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:59.192029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:59.192555 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:59.192755 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:59.192764 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:59.194032 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:59.194038 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:59.195270 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:59.195285 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:59.196176 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:59.196181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:59.196358 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:59.197207 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:59.197229 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:59.201117 (+ 3888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":469,"threads_started":1}
W20260806 22:34:59.201889 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:59.190857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:59289 (local address 127.15.198.190:34327)
0806 22:34:59.191192 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:59.191197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:59.191605 (+ 408us) server_negotiation.cc:407] Connection header received
0806 22:34:59.191783 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:59.191788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:59.191866 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:59.191974 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:59.192931 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:59.193842 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:59.195470 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:59.195996 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:59.196136 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:59.201592 (+ 5456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.131:59289: BlockingRecv error: recv got EOF from 127.15.198.131:59289 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 06 22:34:59 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:34:59 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:34:59.627460 29481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:34:59.617184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:60887)
0806 22:34:59.617850 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0806 22:34:59.617878 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:34:59.618022 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:34:59.618368 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:34:59.618371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:34:59.618906 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:34:59.619202 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:59.619214 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:59.620419 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:59.620423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:59.621410 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:34:59.621422 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:34:59.622315 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:34:59.622321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:34:59.622526 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:59.623366 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:34:59.623388 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:34:59.627050 (+ 3662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":332,"threads_started":1}
W20260806 22:34:59.627630 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:34:59.617401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:60887 (local address 127.15.198.190:34327)
0806 22:34:59.617672 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 22:34:59.617680 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:34:59.617967 (+ 287us) server_negotiation.cc:407] Connection header received
0806 22:34:59.618148 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:34:59.618152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:34:59.618228 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:34:59.618319 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:34:59.619379 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:59.620270 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:59.621600 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:34:59.622144 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:34:59.622297 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:34:59.627330 (+ 5033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:60887: BlockingRecv error: recv got EOF from 127.15.198.130:60887 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 06 22:35:00 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:00 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:00.077785 29483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:00.061366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:50183)
0806 22:35:00.062154 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:00.062187 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:00.062344 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:00.062749 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:00.062753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:00.063525 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:00.063742 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:00.063754 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:00.064945 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:00.064950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:00.065933 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:00.065944 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:00.066741 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:00.066745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:00.066878 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:00.073323 (+ 6445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:00.073361 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:00.077499 (+ 4138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":429,"threads_started":1}
W20260806 22:35:00.078052 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:00.061505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:50183 (local address 127.15.198.190:34327)
0806 22:35:00.061856 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:00.061863 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:00.062274 (+ 411us) server_negotiation.cc:407] Connection header received
0806 22:35:00.062471 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:00.062475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:00.062554 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:00.062646 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:00.063931 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:00.064803 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:00.066114 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:00.066579 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:00.066696 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:00.077743 (+ 11047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:50183: BlockingRecv error: recv got EOF from 127.15.198.129:50183 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 06 22:35:00 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:00 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:00.215266 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:00.203154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:56445)
0806 22:35:00.203794 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:00.203813 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:00.203979 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:00.204637 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:00.204644 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:00.205295 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:00.205521 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:00.205536 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:00.206980 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:00.206987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:00.208419 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:00.208444 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:00.209599 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:00.209610 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:00.209810 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:00.210802 (+ 992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:00.210823 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:00.214824 (+ 4001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":311,"threads_started":1}
W20260806 22:35:00.215466 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:00.203407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:56445 (local address 127.15.198.190:34327)
0806 22:35:00.203725 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:00.203733 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:00.203957 (+ 224us) server_negotiation.cc:407] Connection header received
0806 22:35:00.204199 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:00.204204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:00.204369 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:00.204585 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:00.205725 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:00.206790 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:00.208621 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:00.209411 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:00.209525 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:00.215222 (+ 5697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.131:56445: BlockingRecv error: recv got EOF from 127.15.198.131:56445 (error 108)
Metrics: {"server-negotiator.queue_time_us":140}
Aug 06 22:35:00 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:00 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:00.640810 29486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:00.629008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:44983)
0806 22:35:00.629765 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:00.629795 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:00.629981 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:00.630504 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:00.630509 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:00.631208 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:00.631416 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:00.631429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:00.632749 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:00.632754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:00.634165 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:00.634191 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:00.635319 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:00.635328 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:00.635542 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:00.636478 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:00.636501 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:00.640408 (+ 3907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":457,"threads_started":1}
W20260806 22:35:00.641103 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:00.629247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:44983 (local address 127.15.198.190:34327)
0806 22:35:00.629508 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:00.629514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:00.629926 (+ 412us) server_negotiation.cc:407] Connection header received
0806 22:35:00.630194 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:00.630199 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:00.630282 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:00.630385 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:00.631640 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:00.632538 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:00.634346 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:00.635118 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:00.635256 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:00.640744 (+ 5488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:44983: BlockingRecv error: recv got EOF from 127.15.198.130:44983 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 06 22:35:01 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:01 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:01.090888 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:01.079432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:51419)
0806 22:35:01.080346 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:01.080387 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:01.080608 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:01.081060 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:01.081065 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:01.081647 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:01.081863 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:01.081874 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:01.083519 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:01.083527 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:01.084709 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:01.084726 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:01.085660 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:01.085667 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:01.085880 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:01.086691 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:01.086713 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:01.090407 (+ 3694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":455,"threads_started":1}
W20260806 22:35:01.091104 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:01.079630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:51419 (local address 127.15.198.190:34327)
0806 22:35:01.079943 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:01.079948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:01.080534 (+ 586us) server_negotiation.cc:407] Connection header received
0806 22:35:01.080772 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:01.080777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:01.080863 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:01.080968 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:01.082064 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:01.083260 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:01.084901 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:01.085433 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:01.085595 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:01.090727 (+ 5132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:51419: BlockingRecv error: recv got EOF from 127.15.198.129:51419 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 06 22:35:01 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:01 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:01.230002 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:01.216803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:54713)
0806 22:35:01.217496 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:01.217539 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:01.217731 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:01.218199 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:01.218203 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:01.218715 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:01.218924 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:01.218979 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:01.220822 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:01.220830 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:01.222487 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:01.222504 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:01.223440 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:01.223447 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:01.223656 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:01.224475 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:01.224528 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:01.229578 (+ 5050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":376,"threads_started":1}
W20260806 22:35:01.230916 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:01.216882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:54713 (local address 127.15.198.190:34327)
0806 22:35:01.217118 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:01.217128 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:01.217676 (+ 548us) server_negotiation.cc:407] Connection header received
0806 22:35:01.217907 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:01.217912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:01.217993 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:01.218090 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:01.219261 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:01.220641 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:01.222678 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:01.223243 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:01.223389 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:01.230626 (+ 7237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.131:54713: BlockingRecv error: recv got EOF from 127.15.198.131:54713 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 22:35:01 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:01 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:01.654881 29490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:01.642564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:50155)
0806 22:35:01.643601 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:01.643622 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:01.643802 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:01.644471 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:01.644477 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:01.645325 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:01.645583 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:01.645598 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:01.647087 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:01.647099 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:01.648289 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:01.648311 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:01.649391 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:01.649398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:01.649559 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:01.650500 (+ 941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:01.650536 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:01.654532 (+ 3996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":902,"thread_start_us":561,"threads_started":1}
W20260806 22:35:01.655326 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:01.642555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:50155 (local address 127.15.198.190:34327)
0806 22:35:01.643125 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:01.643135 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:01.643771 (+ 636us) server_negotiation.cc:407] Connection header received
0806 22:35:01.643988 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:01.643993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:01.644116 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:01.644309 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:01.645837 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:01.646749 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:01.648533 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:01.649105 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:01.649353 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:01.654889 (+ 5536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:50155: BlockingRecv error: recv got EOF from 127.15.198.130:50155 (error 108)
Metrics: {"server-negotiator.queue_time_us":184}
Aug 06 22:35:02 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:02 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:02.108093 29491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:02.092762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:34039)
0806 22:35:02.093626 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:02.093665 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:02.093979 (+ 314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:02.094489 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:02.094496 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:02.095127 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:02.095355 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:02.095368 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:02.096658 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:02.096665 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:02.098325 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:02.098357 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:02.100662 (+ 2305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:02.100675 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:02.100948 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:02.102388 (+ 1440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:02.102414 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:02.107593 (+ 5179us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":438,"threads_started":1}
W20260806 22:35:02.108326 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:02.092890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:34039 (local address 127.15.198.190:34327)
0806 22:35:02.093244 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:02.093254 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:02.093832 (+ 578us) server_negotiation.cc:407] Connection header received
0806 22:35:02.094174 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:02.094179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:02.094285 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:02.094419 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:02.095563 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:02.096394 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:02.098997 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:02.100144 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:02.100288 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:02.107963 (+ 7675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:34039: BlockingRecv error: recv got EOF from 127.15.198.129:34039 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Aug 06 22:35:02 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:02 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:35:02.245045 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:02.232412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:60849 (local address 127.15.198.190:34327)
0806 22:35:02.232696 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:02.232706 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:02.232965 (+ 259us) server_negotiation.cc:407] Connection header received
0806 22:35:02.233635 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:02.233641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:02.233742 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:02.233861 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:02.235111 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:02.236249 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:02.237977 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:02.238472 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:02.238609 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:02.244818 (+ 6209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.131:60849: BlockingRecv error: recv got EOF from 127.15.198.131:60849 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 22:35:02.245170 29492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:02.231962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:60849)
0806 22:35:02.232781 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:02.232804 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:02.233077 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:02.233951 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:02.233955 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:02.234617 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:02.234853 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:02.234864 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:02.236469 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:02.236475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:02.237759 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:02.237779 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:02.238765 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:02.238774 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:02.239073 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:02.239880 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:02.239912 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:02.244529 (+ 4617us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":376,"threads_started":1}
Aug 06 22:35:02 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:02 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:02.669255 29493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:02.656529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:39613)
0806 22:35:02.657304 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:02.657327 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:02.657508 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:02.658081 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:02.658086 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:02.658636 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:02.658862 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:02.658873 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:02.660592 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:02.660597 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:02.661757 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:02.661780 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:02.662952 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:02.662959 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:02.663220 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:02.664180 (+ 960us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:02.664205 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:02.668940 (+ 4735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":410,"threads_started":1}
W20260806 22:35:02.669641 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:02.656704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:39613 (local address 127.15.198.190:34327)
0806 22:35:02.657050 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:02.657055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:02.657499 (+ 444us) server_negotiation.cc:407] Connection header received
0806 22:35:02.657704 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:02.657709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:02.657796 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:02.657911 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:02.659153 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:02.660366 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:02.662078 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:02.662636 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:02.662779 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:02.669288 (+ 6509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:39613: BlockingRecv error: recv got EOF from 127.15.198.130:39613 (error 108)
Metrics: {"server-negotiator.queue_time_us":186}
Aug 06 22:35:03 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:03 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:03.124349 29494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:03.110213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:40601)
0806 22:35:03.111281 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:03.111317 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:03.111519 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:03.112205 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:03.112209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:03.112794 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:03.113020 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:03.113039 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:03.114507 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:03.114512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:03.116070 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:03.116091 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:03.117482 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:03.117493 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:03.117883 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:03.118854 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:03.118878 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:03.123807 (+ 4929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":895,"thread_start_us":594,"threads_started":1}
W20260806 22:35:03.124455 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:03.110263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:40601 (local address 127.15.198.190:34327)
0806 22:35:03.110557 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:03.110569 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:03.111440 (+ 871us) server_negotiation.cc:407] Connection header received
0806 22:35:03.111688 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:03.111695 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:03.111812 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:03.111893 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:03.113282 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:03.114311 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:03.116311 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:03.117102 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:03.117327 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:03.124161 (+ 6834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:40601: BlockingRecv error: recv got EOF from 127.15.198.129:40601 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 06 22:35:03 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:03 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:03.259922 29495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:03.247415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:52021)
0806 22:35:03.248144 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:03.248165 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:03.248378 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:03.248966 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:03.248972 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:03.249598 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:03.249881 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:03.249902 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:03.252021 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:03.252027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:03.253223 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:03.253245 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:03.254317 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:03.254324 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:03.254492 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:03.255458 (+ 966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:03.255493 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:03.259617 (+ 4124us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":379,"threads_started":1}
W20260806 22:35:03.260427 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:03.247526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:52021 (local address 127.15.198.190:34327)
0806 22:35:03.247840 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:03.247849 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:03.248300 (+ 451us) server_negotiation.cc:407] Connection header received
0806 22:35:03.248601 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:03.248610 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:03.248716 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:03.248844 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:03.250105 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:03.251723 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:03.253467 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:03.253996 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:03.254130 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:03.260078 (+ 5948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.131:52021: BlockingRecv error: recv got EOF from 127.15.198.131:52021 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 06 22:35:03 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:03 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:03.684787 29496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:03.671182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:37971)
0806 22:35:03.671946 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:03.671966 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:03.672177 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:03.672617 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:03.672622 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:03.673174 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:03.673385 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:03.673396 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:03.674548 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:03.674555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:03.675688 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:03.675704 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:03.676585 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:03.676592 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:03.676807 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:03.677830 (+ 1023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:03.677855 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:03.684337 (+ 6482us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":408,"threads_started":1}
W20260806 22:35:03.684900 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:03.671180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:37971 (local address 127.15.198.190:34327)
0806 22:35:03.671484 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:03.671491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:03.672081 (+ 590us) server_negotiation.cc:407] Connection header received
0806 22:35:03.672332 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:03.672337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:03.672437 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:03.672574 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:03.673560 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:03.674357 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:03.675853 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:03.676379 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:03.676551 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:03.684552 (+ 8001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:37971: BlockingRecv error: recv got EOF from 127.15.198.130:37971 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 06 22:35:04 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:04 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:04.141271 29497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:04.126086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:55701)
0806 22:35:04.126890 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:04.126932 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:04.127229 (+ 297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:04.127803 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:04.127809 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:04.128530 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:04.128783 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:04.128796 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:04.131662 (+ 2866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:04.131668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:04.133463 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:04.133494 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:04.134582 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:04.134589 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:04.134754 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:04.135720 (+ 966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:04.135745 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:04.140723 (+ 4978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":466,"threads_started":1}
W20260806 22:35:04.141763 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:04.126501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:55701 (local address 127.15.198.190:34327)
0806 22:35:04.126772 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:04.126780 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:04.127112 (+ 332us) server_negotiation.cc:407] Connection header received
0806 22:35:04.127423 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:04.127429 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:04.127574 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:04.127688 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:04.129553 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:04.131396 (+ 1843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:04.133776 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:04.134317 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:04.134473 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:04.141211 (+ 6738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:55701: BlockingRecv error: recv got EOF from 127.15.198.129:55701 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
Aug 06 22:35:04 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:04 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:04.273588 29499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:04.261376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:36415)
0806 22:35:04.262264 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:04.262285 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:04.262543 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:04.262985 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:04.262992 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:04.263607 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:04.263820 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:04.263832 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:04.265103 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:04.265118 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:04.266306 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:04.266324 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:04.267435 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:04.267443 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:04.267678 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:04.268481 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:04.268505 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:04.273152 (+ 4647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":741,"thread_start_us":543,"threads_started":1}
W20260806 22:35:04.273733 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:04.261406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:36415 (local address 127.15.198.190:34327)
0806 22:35:04.261672 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:04.261683 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:04.262443 (+ 760us) server_negotiation.cc:407] Connection header received
0806 22:35:04.262716 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:04.262722 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:04.262801 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:04.262932 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:04.264003 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:04.264920 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:04.266531 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:04.267222 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:04.267390 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:04.273438 (+ 6048us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.131:36415: BlockingRecv error: recv got EOF from 127.15.198.131:36415 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 22:35:04 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:04 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:04.699826 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:04.686433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:45405)
0806 22:35:04.687295 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:04.687317 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:04.687517 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:04.688027 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:04.688031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:04.688539 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:04.688740 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:04.688751 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:04.690129 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:04.690135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:04.691555 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:04.691578 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:04.693182 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:04.693189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:04.693396 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:04.694513 (+ 1117us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:04.694560 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:04.699195 (+ 4635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":535,"threads_started":1}
W20260806 22:35:04.700037 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:04.686598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:45405 (local address 127.15.198.190:34327)
0806 22:35:04.687138 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:04.687160 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:04.687487 (+ 327us) server_negotiation.cc:407] Connection header received
0806 22:35:04.687689 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:04.687694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:04.687778 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:04.687891 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:04.689001 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:04.689909 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:04.691982 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:04.692911 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:04.693096 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:04.699700 (+ 6604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:45405: BlockingRecv error: recv got EOF from 127.15.198.130:45405 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 06 22:35:05 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:05 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:05.159308 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:05.142671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:39753)
0806 22:35:05.143362 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:05.143394 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:05.143561 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:05.143972 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:05.143978 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:05.144574 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:05.144920 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:05.144939 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:05.146198 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:05.146202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:05.147285 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:05.147301 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:05.148263 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:05.148268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:05.148439 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:05.153368 (+ 4929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:05.153398 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:05.158876 (+ 5478us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":391,"threads_started":1}
W20260806 22:35:05.159571 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:05.143024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:39753 (local address 127.15.198.190:34327)
0806 22:35:05.143294 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:05.143300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:05.143513 (+ 213us) server_negotiation.cc:407] Connection header received
0806 22:35:05.143685 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:05.143688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:05.143780 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:05.143877 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:05.145165 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:05.146008 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:05.147494 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:05.148045 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:05.148172 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:05.159258 (+ 11086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:39753: BlockingRecv error: recv got EOF from 127.15.198.129:39753 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 06 22:35:05 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:05 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:05.287612 29505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:05.275319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:37885)
0806 22:35:05.276061 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:05.276083 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:05.276319 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:05.276888 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:05.276894 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:05.277820 (+ 926us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:05.278049 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:05.278064 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:05.279519 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:05.279527 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:05.280735 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:05.280753 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:05.281659 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:05.281666 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:05.281872 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:05.283020 (+ 1148us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:05.283148 (+ 128us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:05.287293 (+ 4145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":434,"threads_started":1}
W20260806 22:35:05.288049 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:05.275456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:37885 (local address 127.15.198.190:34327)
0806 22:35:05.275834 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:05.275843 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:05.276216 (+ 373us) server_negotiation.cc:407] Connection header received
0806 22:35:05.276497 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:05.276502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:05.276592 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:05.276861 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:05.278243 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:05.279346 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:05.280917 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:05.281467 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:05.281627 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:05.287654 (+ 6027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.131:37885: BlockingRecv error: recv got EOF from 127.15.198.131:37885 (error 108)
Metrics: {"server-negotiator.queue_time_us":148}
Aug 06 22:35:05 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:05 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:05.715415 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:05.701771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:46413)
0806 22:35:05.702578 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:05.702597 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:05.702804 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:05.703314 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:05.703323 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:05.704037 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:05.704375 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:05.704387 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:05.705772 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:05.705776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:05.707063 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:05.707082 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:05.708028 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:05.708034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:05.708271 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:05.709842 (+ 1571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:05.709873 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:05.714934 (+ 5061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":488,"threads_started":1}
W20260806 22:35:05.715979 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:05.702075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:46413 (local address 127.15.198.190:34327)
0806 22:35:05.702293 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:05.702298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:05.702735 (+ 437us) server_negotiation.cc:407] Connection header received
0806 22:35:05.702986 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:05.702990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:05.703124 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:05.703215 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:05.704737 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:05.705567 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:05.707278 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:05.707812 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:05.707946 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:05.715633 (+ 7687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:46413: BlockingRecv error: recv got EOF from 127.15.198.130:46413 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 22:35:06 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:06 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:06.173491 29508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:06.160875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:60831)
0806 22:35:06.161706 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:06.161734 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:06.161905 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:06.162671 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:06.162676 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:06.163306 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:06.163544 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:06.163555 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:06.165209 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:06.165214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:06.166495 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:06.166513 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:06.167779 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:06.167786 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:06.168010 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:06.168903 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:06.168926 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:06.173163 (+ 4237us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":531,"threads_started":1}
W20260806 22:35:06.174288 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:06.161109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:60831 (local address 127.15.198.190:34327)
0806 22:35:06.161366 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:06.161372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:06.161855 (+ 483us) server_negotiation.cc:407] Connection header received
0806 22:35:06.162143 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:06.162148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:06.162288 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:06.162397 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:06.163773 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:06.164886 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:06.166673 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:06.167481 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:06.167703 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:06.173994 (+ 6291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:60831: BlockingRecv error: recv got EOF from 127.15.198.129:60831 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 06 22:35:06 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:06 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:35:06.303398 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:06.289402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:38935 (local address 127.15.198.190:34327)
0806 22:35:06.289838 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:06.289846 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:06.290300 (+ 454us) server_negotiation.cc:407] Connection header received
0806 22:35:06.290573 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:06.290577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:06.290656 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:06.290778 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:06.291948 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:06.293088 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:06.294622 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:06.295226 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:06.295389 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:06.303138 (+ 7749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.131:38935: BlockingRecv error: recv got EOF from 127.15.198.131:38935 (error 108)
Metrics: {"server-negotiator.queue_time_us":177}
I20260806 22:35:06.303403 29509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:06.289067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:38935)
0806 22:35:06.290151 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:06.290172 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:06.290401 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:06.290821 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:06.290827 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:06.291505 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:06.291745 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:06.291756 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:06.293321 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:06.293329 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:06.294410 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:06.294429 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:06.295436 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:06.295444 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:06.296103 (+ 659us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:06.297622 (+ 1519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:06.297656 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:06.302667 (+ 5011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":945,"thread_start_us":543,"threads_started":1}
Aug 06 22:35:06 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:06 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:06.732079 29511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:06.717448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:37483)
0806 22:35:06.718198 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:06.718218 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:06.718393 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:06.718860 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:06.718865 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:06.719716 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:06.719983 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:06.719996 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:06.722349 (+ 2353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:06.722356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:06.723658 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:06.723686 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:06.725211 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:06.725226 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:06.725497 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:06.726597 (+ 1100us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:06.726636 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:06.731738 (+ 5102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":438,"threads_started":1}
W20260806 22:35:06.732765 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:06.717712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:37483 (local address 127.15.198.190:34327)
0806 22:35:06.717967 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:06.717972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:06.718412 (+ 440us) server_negotiation.cc:407] Connection header received
0806 22:35:06.718547 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:06.718552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:06.718634 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:06.718740 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:06.720237 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:06.722016 (+ 1779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:06.723974 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:06.724713 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:06.724941 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:06.732469 (+ 7528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:37483: BlockingRecv error: recv got EOF from 127.15.198.130:37483 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 06 22:35:07 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:07 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:07.187886 29512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:07.175347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:40683)
0806 22:35:07.176156 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:07.176187 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:07.176378 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:07.176962 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:07.176966 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:07.177768 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:07.178078 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:07.178091 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:07.179564 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:07.179569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:07.181025 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:07.181046 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:07.182282 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:07.182288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:07.182495 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:07.183451 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:07.183501 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:07.187559 (+ 4058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":434,"threads_started":1}
W20260806 22:35:07.188287 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:07.175487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:40683 (local address 127.15.198.190:34327)
0806 22:35:07.175753 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:07.175761 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:07.176337 (+ 576us) server_negotiation.cc:407] Connection header received
0806 22:35:07.176603 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:07.176609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:07.176733 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:07.176866 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:07.178265 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:07.179366 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:07.181442 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:07.181987 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:07.182127 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:07.187909 (+ 5782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:40683: BlockingRecv error: recv got EOF from 127.15.198.129:40683 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 06 22:35:07 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:07 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:07.317893 29514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:07.304832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:36491)
0806 22:35:07.305553 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:07.305574 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:07.305741 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:07.306386 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:07.306391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:07.307400 (+ 1009us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:07.307626 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:07.307638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:07.309119 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:07.309124 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:07.310471 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:07.310491 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:07.311555 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:07.311560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:07.311737 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:07.312996 (+ 1259us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:07.313038 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:07.317558 (+ 4520us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":411,"threads_started":1}
W20260806 22:35:07.318284 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:07.304942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:36491 (local address 127.15.198.190:34327)
0806 22:35:07.305233 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:07.305244 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:07.305695 (+ 451us) server_negotiation.cc:407] Connection header received
0806 22:35:07.305887 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:07.305923 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:07.306045 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:07.306176 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:07.307869 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:07.308905 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:07.310678 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:07.311345 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:07.311486 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:07.317835 (+ 6349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.131:36491: BlockingRecv error: recv got EOF from 127.15.198.131:36491 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 06 22:35:07 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:07 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:07.746711 29515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:07.733798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:53967)
0806 22:35:07.734876 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:07.734898 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:07.735123 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:07.735548 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:07.735553 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:07.736096 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:07.736311 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:07.736322 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:07.737551 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:07.737564 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:07.738737 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:07.738761 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:07.739817 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:07.739825 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:07.740077 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:07.740961 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:07.740987 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:07.746189 (+ 5202us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":959,"thread_start_us":464,"threads_started":1}
W20260806 22:35:07.746888 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:07.734192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:53967 (local address 127.15.198.190:34327)
0806 22:35:07.734529 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:07.734536 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:07.735046 (+ 510us) server_negotiation.cc:407] Connection header received
0806 22:35:07.735288 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:07.735294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:07.735385 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:07.735504 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:07.736488 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:07.737350 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:07.738927 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:07.739560 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:07.739728 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:07.746514 (+ 6786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:53967: BlockingRecv error: recv got EOF from 127.15.198.130:53967 (error 108)
Metrics: {"server-negotiator.queue_time_us":165}
Aug 06 22:35:08 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:08 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:08.201472 29517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:08.189265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:46019)
0806 22:35:08.189933 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:08.189968 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:08.190136 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:08.190591 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:08.190595 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:08.191245 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:08.191547 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:08.191558 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:08.192861 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:08.192868 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:08.194165 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:08.194182 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:08.195628 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:08.195643 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:08.195893 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:08.196850 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:08.196873 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:08.201148 (+ 4275us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":390,"threads_started":1}
W20260806 22:35:08.201987 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:08.189490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:46019 (local address 127.15.198.190:34327)
0806 22:35:08.189719 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:08.189724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:08.190099 (+ 375us) server_negotiation.cc:407] Connection header received
0806 22:35:08.190317 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:08.190322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:08.190403 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:08.190521 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:08.191763 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:08.192661 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:08.194400 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:08.195317 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:08.195502 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:08.201525 (+ 6023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:46019: BlockingRecv error: recv got EOF from 127.15.198.129:46019 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 06 22:35:08 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:08 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:08.330691 29518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:08.319316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:49171)
0806 22:35:08.320117 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:08.320138 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:08.320355 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:08.320789 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:08.320794 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:08.321384 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:08.321594 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:08.321605 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:08.322858 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:08.322867 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:08.324120 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:08.324137 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:08.325230 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:08.325238 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:08.325462 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:08.326311 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:08.326333 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:08.330275 (+ 3942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":399,"threads_started":1}
W20260806 22:35:08.330901 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:08.319356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:49171 (local address 127.15.198.190:34327)
0806 22:35:08.319764 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:08.319774 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:08.320262 (+ 488us) server_negotiation.cc:407] Connection header received
0806 22:35:08.320522 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:08.320529 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:08.320617 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:08.320744 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:08.321776 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:08.322679 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:08.324384 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:08.324952 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:08.325142 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:08.330608 (+ 5466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.131:49171: BlockingRecv error: recv got EOF from 127.15.198.131:49171 (error 108)
Metrics: {"server-negotiator.queue_time_us":208}
Aug 06 22:35:08 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:08 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:08.760560 29519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:08.748479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:53149)
0806 22:35:08.749258 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:08.749284 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:08.749482 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:08.749878 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:08.749883 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:08.750727 (+ 844us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:08.751135 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:08.751160 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:08.752476 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:08.752482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:08.753894 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:08.753916 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:08.754968 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:08.754975 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:08.755254 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:08.756116 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:08.756139 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:08.760088 (+ 3949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":492,"threads_started":1}
W20260806 22:35:08.760890 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:08.748611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:53149 (local address 127.15.198.190:34327)
0806 22:35:08.748969 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:08.748977 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:08.749394 (+ 417us) server_negotiation.cc:407] Connection header received
0806 22:35:08.749635 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:08.749640 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:08.749726 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:08.749842 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:08.751349 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:08.752280 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:08.754136 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:08.754680 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:08.754865 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:08.760476 (+ 5611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:53149: BlockingRecv error: recv got EOF from 127.15.198.130:53149 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 06 22:35:09 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:09 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.129@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:09.213825 29520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:09.203443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.129:42571)
0806 22:35:09.204225 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:09.204274 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:09.204437 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:09.204916 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:09.204920 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:09.205505 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:09.205711 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:09.205722 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:09.206964 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:09.206969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:09.208067 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:09.208083 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:09.208889 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:09.208895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:09.209038 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:09.209943 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:09.209965 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:09.213504 (+ 3539us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":440,"threads_started":1}
W20260806 22:35:09.214196 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:09.203593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:42571 (local address 127.15.198.190:34327)
0806 22:35:09.203947 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:09.203953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:09.204411 (+ 458us) server_negotiation.cc:407] Connection header received
0806 22:35:09.204615 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:09.204620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:09.204724 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:09.204826 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:09.205914 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:09.206777 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:09.208222 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:09.208718 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:09.208832 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:09.213824 (+ 4992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.129:42571: BlockingRecv error: recv got EOF from 127.15.198.129:42571 (error 108)
Metrics: {"server-negotiator.queue_time_us":178}
Aug 06 22:35:09 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:09 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.131@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:09.343863 29521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:09.332376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.131:55037)
0806 22:35:09.333029 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:09.333050 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:09.333218 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:09.333613 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:09.333617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:09.334143 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:09.334369 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:09.334381 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:09.335825 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:09.335830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:09.337074 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:09.337092 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:09.338067 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:09.338073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:09.338225 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:09.339293 (+ 1068us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:09.339317 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:09.343491 (+ 4174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":382,"threads_started":1}
W20260806 22:35:09.344645 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:09.332649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:55037 (local address 127.15.198.190:34327)
0806 22:35:09.332970 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:09.332977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:09.333183 (+ 206us) server_negotiation.cc:407] Connection header received
0806 22:35:09.333347 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:09.333352 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:09.333432 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:09.333524 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:09.334552 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:09.335619 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:09.337261 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:09.337795 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:09.337977 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:09.344278 (+ 6301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.131:55037: BlockingRecv error: recv got EOF from 127.15.198.131:55037 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 06 22:35:09 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:35:09 dist-test-slave-h3ft krb5kdc[27996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.198.130@KRBTEST.COM for oryx/127.15.198.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:35:09.774174 29522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:09.762625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:34327 (local address 127.15.198.130:53025)
0806 22:35:09.763459 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:09.763478 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:09.763703 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:09.764116 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:09.764121 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:09.764705 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:09.764933 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:09.764944 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:09.766504 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:09.766514 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:09.767622 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:09.767637 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:09.768444 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:09.768452 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:09.768656 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:09.769722 (+ 1066us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:09.769745 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:09.773709 (+ 3964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.198.190:34327: Server oryx/127.15.198.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":498,"threads_started":1}
W20260806 22:35:09.774343 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:09.762792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:53025 (local address 127.15.198.190:34327)
0806 22:35:09.763187 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:09.763197 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:09.763606 (+ 409us) server_negotiation.cc:407] Connection header received
0806 22:35:09.763874 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:09.763879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:09.763960 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:09.764081 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:09.765095 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:09.766345 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:09.767791 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:09.768264 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:09.768422 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:09.774057 (+ 5635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.198.130:53025: BlockingRecv error: recv got EOF from 127.15.198.130:53025 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
I20260806 22:35:10.126920 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 28840
I20260806 22:35:10.143497 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:10.128758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59152 (local address 127.15.198.129:44831)
0806 22:35:10.129569 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:10.129578 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:10.129612 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:35:10.129693 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:10.129701 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:10.130017 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:10.130210 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:10.132344 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:10.133611 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:10.135120 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:10.135786 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:10.135905 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:10.139495 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:10.139507 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:10.139527 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:35:10.139593 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:10.142745 (+ 3152us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.198.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 22:35:10.143187 (+ 442us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59152: Request ticket server kudu/127.15.198.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":431,"threads_started":1}
W20260806 22:35:10.143656 29524 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59152: Request ticket server kudu/127.15.198.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 22:35:10.143695 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 28840: Not authorized: Client connection negotiation failed: client connection to 127.15.198.129:44831: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.198.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 22:35:10.144006 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 28840
I20260806 22:35:10.184743 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 29072
I20260806 22:35:10.206568 29525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:10.186330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42026 (local address 127.15.198.130:35125)
0806 22:35:10.187141 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:10.187151 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:10.187194 (+ 43us) server_negotiation.cc:407] Connection header received
0806 22:35:10.187276 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:10.187284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:10.187632 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:10.187823 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:10.189359 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:10.190651 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:10.192188 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:10.192921 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:10.193039 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:10.202114 (+ 9075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:10.202131 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:10.202153 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:35:10.202229 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:10.205842 (+ 3613us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.198.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 22:35:10.206247 (+ 405us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42026: Request ticket server kudu/127.15.198.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":348,"threads_started":1}
W20260806 22:35:10.206765 29525 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42026: Request ticket server kudu/127.15.198.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 22:35:10.206923 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 29072: Not authorized: Client connection negotiation failed: client connection to 127.15.198.130:35125: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.198.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 22:35:10.207124 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 29072
I20260806 22:35:10.243144 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 29270
I20260806 22:35:10.259876 29526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:10.244607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38578 (local address 127.15.198.131:44485)
0806 22:35:10.245330 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:10.245339 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:10.245373 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:35:10.245461 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:10.245469 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:10.245765 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:10.245972 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:10.247753 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:10.249033 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:10.250916 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:10.251796 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:10.251933 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:10.255548 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:10.255561 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:10.255584 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:35:10.255640 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:10.259154 (+ 3514us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.198.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 22:35:10.259552 (+ 398us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38578: Request ticket server kudu/127.15.198.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":367,"threads_started":1}
W20260806 22:35:10.260043 29526 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38578: Request ticket server kudu/127.15.198.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 22:35:10.260149 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 29270: Not authorized: Client connection negotiation failed: client connection to 127.15.198.131:44485: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.198.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 22:35:10.260396 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 29270
I20260806 22:35:10.293155 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 28742
I20260806 22:35:10.295388 28832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:35:10.440487 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 28742
2026-08-06T22:35:10Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (23124 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T22:35:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:35:10Z Disabled control of system clock
I20260806 22:35:10.571297 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:35051
--webserver_interface=127.15.198.190
--webserver_port=0
--builtin_ntp_servers=127.15.198.148:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:35051
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 22:35:11.029963 29535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:35:11.030385 29535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:35:11.030555 29535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:35:11.038770 29535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:35:11.038929 29535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:35:11.039018 29535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:35:11.039150 29535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:35:11.039214 29535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:35:11.050082 29535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:35051
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:35051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:35:11.052093 29535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:35:11.054747 29535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:35:11.070515 29541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:11.070643 29544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:11.070619 29546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:35:11.071753 29535 server_base.cc:1034] running on GCE node
I20260806 22:35:11.072964 29535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:35:11.075176 29535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:35:11.076462 29535 hybrid_clock.cc:648] HybridClock initialized: now 1786055711076416 us; error 55 us; skew 500 ppm
I20260806 22:35:11.076941 29535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:35:11.080596 29535 webserver.cc:460] Webserver started at http://127.15.198.190:39697/ using document root <none> and password file <none>
I20260806 22:35:11.081669 29535 fs_manager.cc:359] Metadata directory not provided
I20260806 22:35:11.081843 29535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:35:11.082306 29535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:35:11.086464 29535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "937c3f7bfcb549d88f38ac1ec79914bf"
format_stamp: "Formatted at 2026-08-06 22:35:11 on dist-test-slave-h3ft"
server_key: "ac5c6182bbbe406b7d2987bb3f5d6249"
server_key_iv: "a1d300edb3f505231f811ddb7e17b85a"
server_key_version: "encryptionkey@0"
I20260806 22:35:11.087626 29535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "937c3f7bfcb549d88f38ac1ec79914bf"
format_stamp: "Formatted at 2026-08-06 22:35:11 on dist-test-slave-h3ft"
server_key: "ac5c6182bbbe406b7d2987bb3f5d6249"
server_key_iv: "a1d300edb3f505231f811ddb7e17b85a"
server_key_version: "encryptionkey@0"
I20260806 22:35:11.094955 29535 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260806 22:35:11.100631 29552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:11.102680 29535 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.004s
I20260806 22:35:11.103463 29535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "937c3f7bfcb549d88f38ac1ec79914bf"
format_stamp: "Formatted at 2026-08-06 22:35:11 on dist-test-slave-h3ft"
server_key: "ac5c6182bbbe406b7d2987bb3f5d6249"
server_key_iv: "a1d300edb3f505231f811ddb7e17b85a"
server_key_version: "encryptionkey@0"
I20260806 22:35:11.103873 29535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:35:11.129976 29535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:35:11.131364 29535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:35:11.131750 29535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:35:11.155366 29535 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:35051
I20260806 22:35:11.155393 29642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:35051 every 8 connection(s)
I20260806 22:35:11.157255 29535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:35:11.162537 29643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:35:11.165036 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 29535
I20260806 22:35:11.165355 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:35:11.166205 16154 external_mini_cluster.cc:1428] Setting key 86764ba891946a415703ad9115774863
I20260806 22:35:11.177938 29643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf: Bootstrap starting.
I20260806 22:35:11.182598 29643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf: Neither blocks nor log segments found. Creating new log.
I20260806 22:35:11.184204 29643 log.cc:824] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf: Log is configured to *not* fsync() on all Append() calls
I20260806 22:35:11.184635 29646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:11.169280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58884 (local address 127.15.198.190:35051)
0806 22:35:11.170724 (+ 1444us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:11.170739 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:11.170803 (+ 64us) server_negotiation.cc:407] Connection header received
0806 22:35:11.173181 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:11.173241 (+ 60us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:11.174022 (+ 781us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:11.174662 (+ 640us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:11.175587 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:11.177050 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:11.178703 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:11.179622 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:11.179870 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:11.181525 (+ 1655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:11.181536 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:11.181556 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:35:11.182290 (+ 734us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:11.182410 (+ 120us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:35:11.182446 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:11.182629 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:11.182877 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 22:35:11.183348 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":820,"thread_start_us":486,"threads_started":1}
I20260806 22:35:11.188402 29643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf: No bootstrap required, opened a new log
I20260806 22:35:11.197393 29643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937c3f7bfcb549d88f38ac1ec79914bf" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 35051 } }
I20260806 22:35:11.197932 29643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:35:11.198058 29643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937c3f7bfcb549d88f38ac1ec79914bf, State: Initialized, Role: FOLLOWER
I20260806 22:35:11.198907 29643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937c3f7bfcb549d88f38ac1ec79914bf" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 35051 } }
I20260806 22:35:11.199295 29643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:35:11.199469 29643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:35:11.199699 29643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:35:11.202363 29643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937c3f7bfcb549d88f38ac1ec79914bf" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 35051 } }
I20260806 22:35:11.203183 29643 leader_election.cc:302] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 937c3f7bfcb549d88f38ac1ec79914bf; no voters:
I20260806 22:35:11.204063 29643 leader_election.cc:288] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:35:11.204320 29650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:35:11.205687 29650 raft_consensus.cc:686] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [term 1 LEADER]: Becoming Leader. State: Replica: 937c3f7bfcb549d88f38ac1ec79914bf, State: Running, Role: LEADER
I20260806 22:35:11.206558 29650 consensus_queue.cc:237] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937c3f7bfcb549d88f38ac1ec79914bf" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 35051 } }
I20260806 22:35:11.206965 29643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:35:11.210757 29653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 937c3f7bfcb549d88f38ac1ec79914bf. Latest consensus state: current_term: 1 leader_uuid: "937c3f7bfcb549d88f38ac1ec79914bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937c3f7bfcb549d88f38ac1ec79914bf" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 35051 } } }
I20260806 22:35:11.211524 29653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [sys.catalog]: This master's current role is: LEADER
I20260806 22:35:11.210815 29651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "937c3f7bfcb549d88f38ac1ec79914bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937c3f7bfcb549d88f38ac1ec79914bf" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 35051 } } }
I20260806 22:35:11.212313 29651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf [sys.catalog]: This master's current role is: LEADER
I20260806 22:35:11.212725 29664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:35:11.221565 29664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:35:11.235433 29664 catalog_manager.cc:1347] Generated new cluster ID: 91c78e7cc2fa4acbafaa46f712f26672
I20260806 22:35:11.235632 29664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:35:11.258778 29664 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:35:11.261655 29664 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:35:11.275967 29664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 937c3f7bfcb549d88f38ac1ec79914bf: Generated new TSK 0
I20260806 22:35:11.277467 29664 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:35:11.289089 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:35051
--builtin_ntp_servers=127.15.198.148:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 22:35:11.771709 29676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:35:11.772164 29676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:35:11.772308 29676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:35:11.781520 29676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:35:11.781682 29676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:35:11.781879 29676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:35:11.794490 29676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:35051
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:35:11.796540 29676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:35:11.799731 29676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:35:11.818500 29685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:11.818696 29682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:11.818470 29683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:35:11.820518 29676 server_base.cc:1034] running on GCE node
I20260806 22:35:11.821743 29676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:35:11.823648 29676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:35:11.825026 29676 hybrid_clock.cc:648] HybridClock initialized: now 1786055711824955 us; error 93 us; skew 500 ppm
I20260806 22:35:11.825498 29676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:35:11.828794 29676 webserver.cc:460] Webserver started at http://127.15.198.129:33641/ using document root <none> and password file <none>
I20260806 22:35:11.829785 29676 fs_manager.cc:359] Metadata directory not provided
I20260806 22:35:11.829942 29676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:35:11.830394 29676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:35:11.834209 29676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "3cc466764a0945b7b6e8207cd8464354"
format_stamp: "Formatted at 2026-08-06 22:35:11 on dist-test-slave-h3ft"
server_key: "e0bf2416eaa4c87f27c1094bae38ef94"
server_key_iv: "5f4e3e3d1fb2f606043971e7cd8b8e39"
server_key_version: "encryptionkey@0"
I20260806 22:35:11.835568 29676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "3cc466764a0945b7b6e8207cd8464354"
format_stamp: "Formatted at 2026-08-06 22:35:11 on dist-test-slave-h3ft"
server_key: "e0bf2416eaa4c87f27c1094bae38ef94"
server_key_iv: "5f4e3e3d1fb2f606043971e7cd8b8e39"
server_key_version: "encryptionkey@0"
I20260806 22:35:11.842810 29676 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 22:35:11.847875 29691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:11.849898 29676 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:35:11.850154 29676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "3cc466764a0945b7b6e8207cd8464354"
format_stamp: "Formatted at 2026-08-06 22:35:11 on dist-test-slave-h3ft"
server_key: "e0bf2416eaa4c87f27c1094bae38ef94"
server_key_iv: "5f4e3e3d1fb2f606043971e7cd8b8e39"
server_key_version: "encryptionkey@0"
I20260806 22:35:11.850436 29676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:35:11.881183 29676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:35:11.882427 29676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:35:11.882824 29676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:35:11.884395 29676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:35:11.886474 29676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:35:11.886626 29676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:11.886786 29676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:35:11.886906 29676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:11.928903 29676 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:39725
I20260806 22:35:11.929049 29857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:39725 every 8 connection(s)
I20260806 22:35:11.930646 29676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:35:11.931957 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 29676
I20260806 22:35:11.932224 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:35:11.933053 16154 external_mini_cluster.cc:1428] Setting key ca950e3cc08ee2550deb23618412c5be
I20260806 22:35:11.937395 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:35051
--builtin_ntp_servers=127.15.198.148:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 22:35:11.951531 29862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:11.934402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:34267 (local address 127.15.198.190:35051)
0806 22:35:11.935229 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:11.935236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:11.937054 (+ 1818us) server_negotiation.cc:407] Connection header received
0806 22:35:11.940543 (+ 3489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:11.940549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:11.940667 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:11.940802 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:11.942822 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:11.944054 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:11.947193 (+ 3139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:11.947688 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:11.947820 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:11.950091 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:11.950099 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:11.950104 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:35:11.950138 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:11.950204 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:35:11.950222 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:11.950570 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:11.950934 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 22:35:11.951160 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":545,"threads_started":1}
I20260806 22:35:11.953456 29863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:11.935324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:35051 (local address 127.15.198.129:34267)
0806 22:35:11.936820 (+ 1496us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:11.936883 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:11.940078 (+ 3195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:11.941204 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:11.941223 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:11.941409 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:11.942534 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:11.942571 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:11.944306 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:11.944315 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:11.946858 (+ 2543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:11.946885 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:11.948008 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:11.948019 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:11.948451 (+ 432us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:11.949540 (+ 1089us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:35:11.949547 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:11.949638 (+ 91us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:35:11.949800 (+ 162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:35:11.950571 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:35:11.950586 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:35:11.950601 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 22:35:11.950870 (+ 269us) client_negotiation.cc:240] Negotiation successful
0806 22:35:11.951554 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"thread_start_us":415,"threads_started":1}
I20260806 22:35:11.956116 29859 heartbeater.cc:347] Connected to a master server at 127.15.198.190:35051
I20260806 22:35:11.956589 29859 heartbeater.cc:464] Registering TS with master...
I20260806 22:35:11.957815 29859 heartbeater.cc:511] Master 127.15.198.190:35051 requested a full tablet report, sending...
I20260806 22:35:11.960963 29585 ts_manager.cc:194] Registered new tserver with Master: 3cc466764a0945b7b6e8207cd8464354 (127.15.198.129:39725)
I20260806 22:35:11.962965 29585 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.198.129:34267
W20260806 22:35:12.395190 29864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:35:12.395668 29864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:35:12.395843 29864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:35:12.404162 29864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:35:12.404374 29864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:35:12.404588 29864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:35:12.415719 29864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:35051
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:35:12.417639 29864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:35:12.420252 29864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:35:12.438680 29872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:12.438680 29873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:12.438611 29877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:35:12.440249 29864 server_base.cc:1034] running on GCE node
I20260806 22:35:12.441216 29864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:35:12.442612 29864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:35:12.443893 29864 hybrid_clock.cc:648] HybridClock initialized: now 1786055712443778 us; error 92 us; skew 500 ppm
I20260806 22:35:12.444376 29864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:35:12.447961 29864 webserver.cc:460] Webserver started at http://127.15.198.130:38679/ using document root <none> and password file <none>
I20260806 22:35:12.448977 29864 fs_manager.cc:359] Metadata directory not provided
I20260806 22:35:12.449119 29864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:35:12.449573 29864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:35:12.457585 29864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "617a3a403cac48a3a8d1473d58c26e05"
format_stamp: "Formatted at 2026-08-06 22:35:12 on dist-test-slave-h3ft"
server_key: "a7c6292b1708e9076b01ff739899cdfb"
server_key_iv: "3ea47d31a65e7e623144fee20102e9a9"
server_key_version: "encryptionkey@0"
I20260806 22:35:12.458870 29864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "617a3a403cac48a3a8d1473d58c26e05"
format_stamp: "Formatted at 2026-08-06 22:35:12 on dist-test-slave-h3ft"
server_key: "a7c6292b1708e9076b01ff739899cdfb"
server_key_iv: "3ea47d31a65e7e623144fee20102e9a9"
server_key_version: "encryptionkey@0"
I20260806 22:35:12.466288 29864 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.004s
I20260806 22:35:12.471841 29887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:12.474383 29864 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260806 22:35:12.474759 29864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "617a3a403cac48a3a8d1473d58c26e05"
format_stamp: "Formatted at 2026-08-06 22:35:12 on dist-test-slave-h3ft"
server_key: "a7c6292b1708e9076b01ff739899cdfb"
server_key_iv: "3ea47d31a65e7e623144fee20102e9a9"
server_key_version: "encryptionkey@0"
I20260806 22:35:12.475160 29864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:35:12.498031 29864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:35:12.499411 29864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:35:12.499822 29864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:35:12.501231 29864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:35:12.503333 29864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:35:12.503485 29864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:12.503650 29864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:35:12.503748 29864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:12.549067 29864 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:45097
I20260806 22:35:12.549088 30050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:45097 every 8 connection(s)
I20260806 22:35:12.551966 29864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:35:12.556900 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 29864
I20260806 22:35:12.557176 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:35:12.557739 16154 external_mini_cluster.cc:1428] Setting key 8dec03013d22c32d412bd559b2b3e7d1
I20260806 22:35:12.561921 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:35051
--builtin_ntp_servers=127.15.198.148:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 22:35:12.571120 30053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:12.556160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:58643 (local address 127.15.198.190:35051)
0806 22:35:12.556938 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:12.556948 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:12.558829 (+ 1881us) server_negotiation.cc:407] Connection header received
0806 22:35:12.561523 (+ 2694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:12.561533 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:12.561662 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:12.561806 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:12.563622 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:12.564950 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:12.566945 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:12.567752 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:12.567921 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:12.569850 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:12.569860 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:12.569867 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:35:12.569903 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:12.569970 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:35:12.569990 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:12.570094 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:12.570379 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 22:35:12.570526 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":426,"threads_started":1}
I20260806 22:35:12.572485 30054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:12.557058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:35051 (local address 127.15.198.130:58643)
0806 22:35:12.558601 (+ 1543us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:12.558674 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:12.561125 (+ 2451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:12.562258 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:12.562284 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:12.562426 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:12.563380 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:12.563409 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:12.565184 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:12.565189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:12.566715 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:12.566738 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:12.568089 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:12.568102 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:12.568402 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:12.569401 (+ 999us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:35:12.569404 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:12.569469 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:35:12.569588 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:35:12.570104 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:35:12.570113 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:35:12.570121 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:35:12.570318 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 22:35:12.570773 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"thread_start_us":479,"threads_started":1}
I20260806 22:35:12.574741 30051 heartbeater.cc:347] Connected to a master server at 127.15.198.190:35051
I20260806 22:35:12.575292 30051 heartbeater.cc:464] Registering TS with master...
I20260806 22:35:12.576540 30051 heartbeater.cc:511] Master 127.15.198.190:35051 requested a full tablet report, sending...
I20260806 22:35:12.578855 29585 ts_manager.cc:194] Registered new tserver with Master: 617a3a403cac48a3a8d1473d58c26e05 (127.15.198.130:45097)
I20260806 22:35:12.580029 29585 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.198.130:58643
I20260806 22:35:12.967577 29859 heartbeater.cc:503] Master 127.15.198.190:35051 was elected leader, sending a full tablet report...
W20260806 22:35:13.025105 30056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:35:13.025516 30056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:35:13.025614 30056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:35:13.034124 30056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:35:13.034261 30056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:35:13.034386 30056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:35:13.044868 30056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:35051
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:35:13.046768 30056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:35:13.049410 30056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:35:13.066634 30065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:13.066536 30064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:13.066612 30068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:35:13.067691 30056 server_base.cc:1034] running on GCE node
I20260806 22:35:13.068620 30056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:35:13.069939 30056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:35:13.071193 30056 hybrid_clock.cc:648] HybridClock initialized: now 1786055713071127 us; error 61 us; skew 500 ppm
I20260806 22:35:13.071640 30056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:35:13.074771 30056 webserver.cc:460] Webserver started at http://127.15.198.131:37769/ using document root <none> and password file <none>
I20260806 22:35:13.075906 30056 fs_manager.cc:359] Metadata directory not provided
I20260806 22:35:13.076081 30056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:35:13.076681 30056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:35:13.080853 30056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "f8655947433c4000ad5a9580dcdb23b5"
format_stamp: "Formatted at 2026-08-06 22:35:13 on dist-test-slave-h3ft"
server_key: "dabebf9e1761d40300547197aff70f37"
server_key_iv: "d0a9b991bad89f09a9a483bbb480b644"
server_key_version: "encryptionkey@0"
I20260806 22:35:13.081993 30056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "f8655947433c4000ad5a9580dcdb23b5"
format_stamp: "Formatted at 2026-08-06 22:35:13 on dist-test-slave-h3ft"
server_key: "dabebf9e1761d40300547197aff70f37"
server_key_iv: "d0a9b991bad89f09a9a483bbb480b644"
server_key_version: "encryptionkey@0"
I20260806 22:35:13.089175 30056 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260806 22:35:13.094079 30077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:13.096267 30056 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:35:13.096552 30056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "f8655947433c4000ad5a9580dcdb23b5"
format_stamp: "Formatted at 2026-08-06 22:35:13 on dist-test-slave-h3ft"
server_key: "dabebf9e1761d40300547197aff70f37"
server_key_iv: "d0a9b991bad89f09a9a483bbb480b644"
server_key_version: "encryptionkey@0"
I20260806 22:35:13.096868 30056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:35:13.114886 30056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:35:13.116389 30056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:35:13.116809 30056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:35:13.118199 30056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:35:13.120579 30056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:35:13.120734 30056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:13.120920 30056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:35:13.121028 30056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:13.165887 30056 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:36605
I20260806 22:35:13.165921 30237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:36605 every 8 connection(s)
I20260806 22:35:13.167793 30056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:35:13.177943 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 30056
I20260806 22:35:13.178198 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:35:13.178809 16154 external_mini_cluster.cc:1428] Setting key f09495b43d4bfe292a7e5bbd85dd251d
I20260806 22:35:13.185312 30243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:13.171505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:49689 (local address 127.15.198.190:35051)
0806 22:35:13.172127 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:13.172133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:13.173790 (+ 1657us) server_negotiation.cc:407] Connection header received
0806 22:35:13.176228 (+ 2438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:13.176237 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:13.176383 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:13.176544 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:13.178437 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:13.179561 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:13.181703 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:13.182154 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:13.182304 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:13.184212 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:13.184219 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:13.184222 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:35:13.184244 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:13.184301 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:35:13.184322 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:13.184446 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:13.184733 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 22:35:13.184868 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":348,"threads_started":1}
I20260806 22:35:13.186769 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:13.172260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:35051 (local address 127.15.198.131:49689)
0806 22:35:13.173538 (+ 1278us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:13.173590 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:13.175829 (+ 2239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:13.176904 (+ 1075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:13.176917 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:13.177041 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:13.178150 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:13.178187 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:13.179748 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:13.179759 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:13.181521 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:13.181537 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:13.182438 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:13.182447 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:13.182754 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:13.183727 (+ 973us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:35:13.183731 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:13.183788 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:35:13.183909 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:35:13.184436 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:35:13.184449 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:35:13.184461 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 22:35:13.184718 (+ 257us) client_negotiation.cc:240] Negotiation successful
0806 22:35:13.185290 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":381,"threads_started":1}
I20260806 22:35:13.188683 30240 heartbeater.cc:347] Connected to a master server at 127.15.198.190:35051
I20260806 22:35:13.189101 30240 heartbeater.cc:464] Registering TS with master...
I20260806 22:35:13.190150 30240 heartbeater.cc:511] Master 127.15.198.190:35051 requested a full tablet report, sending...
I20260806 22:35:13.192203 29585 ts_manager.cc:194] Registered new tserver with Master: f8655947433c4000ad5a9580dcdb23b5 (127.15.198.131:36605)
I20260806 22:35:13.193190 29585 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.198.131:49689
I20260806 22:35:13.196009 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:35:13.204054 30243 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:13.201968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58896 (local address 127.15.198.190:35051)
0806 22:35:13.202302 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:13.202308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:13.202750 (+ 442us) server_negotiation.cc:407] Connection header received
0806 22:35:13.203054 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:13.203063 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:13.203156 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:13.203261 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:13.203797 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58896: BlockingRecv error: recv got EOF from 127.0.0.1:58896 (error 108)
Metrics: {"server-negotiator.queue_time_us":138}
I20260806 22:35:13.204840 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 29676
I20260806 22:35:13.216099 30255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:13.206432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40444 (local address 127.15.198.129:39725)
0806 22:35:13.207215 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:13.207224 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:13.207260 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:35:13.207345 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:13.207352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:13.207784 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:13.207953 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:13.208450 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:13.209687 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:13.211599 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:13.212260 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:13.212370 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:13.214053 (+ 1683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:13.214064 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:13.214083 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:35:13.214949 (+ 866us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:13.215097 (+ 148us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:35:13.215130 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:13.215256 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:13.215493 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 22:35:13.215664 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":357,"threads_started":1}
I20260806 22:35:13.217873 29852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:35:13.310300 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 29676
I20260806 22:35:13.340204 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 29864
I20260806 22:35:13.350445 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:13.341665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49276 (local address 127.15.198.130:45097)
0806 22:35:13.342444 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:13.342453 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:13.342487 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:35:13.342567 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:13.342574 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:13.342867 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:13.343086 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:13.343549 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:13.344721 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:13.346144 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:13.346825 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:13.346959 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:13.348529 (+ 1570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:13.348540 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:13.348558 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:35:13.349388 (+ 830us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:13.349457 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:35:13.349482 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:13.349592 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:13.349840 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 22:35:13.350033 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":419,"threads_started":1}
I20260806 22:35:13.352540 30046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:35:13.444641 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 29864
I20260806 22:35:13.476269 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 30056
I20260806 22:35:13.487754 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:13.478004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41990 (local address 127.15.198.131:36605)
0806 22:35:13.479071 (+ 1067us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:13.479097 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:13.479148 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:35:13.479262 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:13.479275 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:13.479725 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:13.479963 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:13.480777 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:13.482083 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:13.483724 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:13.484405 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:13.484520 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:13.485788 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:13.485797 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:13.485818 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:35:13.486606 (+ 788us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:13.486665 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:35:13.486696 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:13.486798 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:13.486988 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 22:35:13.487217 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":537,"threads_started":1}
I20260806 22:35:13.489466 30229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:35:13.582968 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 30056
I20260806 22:35:13.612453 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 29535
I20260806 22:35:13.614248 29636 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:35:13.736413 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 29535
2026-08-06T22:35:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (3284 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T22:35:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:35:13Z Disabled control of system clock
I20260806 22:35:13.856952 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:33763
--webserver_interface=127.15.198.190
--webserver_port=0
--builtin_ntp_servers=127.15.198.148:41557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:33763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:35:14.300240 30269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:35:14.300722 30269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:35:14.300827 30269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:35:14.309720 30269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:35:14.309939 30269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:35:14.309990 30269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:35:14.310056 30269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:35:14.310115 30269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:35:14.321322 30269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:41557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:33763
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:33763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:35:14.323271 30269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:35:14.325717 30269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:35:14.341382 30275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:14.341408 30277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:14.341382 30281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:35:14.342422 30269 server_base.cc:1034] running on GCE node
I20260806 22:35:14.343711 30269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:35:14.345778 30269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:35:14.347112 30269 hybrid_clock.cc:648] HybridClock initialized: now 1786055714347020 us; error 71 us; skew 500 ppm
I20260806 22:35:14.347604 30269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:35:14.351109 30269 webserver.cc:460] Webserver started at http://127.15.198.190:36823/ using document root <none> and password file <none>
I20260806 22:35:14.352125 30269 fs_manager.cc:359] Metadata directory not provided
I20260806 22:35:14.352263 30269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:35:14.352715 30269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:35:14.356634 30269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "f247277e4620467fa75f4bec06937bfa"
format_stamp: "Formatted at 2026-08-06 22:35:14 on dist-test-slave-h3ft"
server_key: "a9d197fc34b6d3bea8ab95c255e05e66"
server_key_iv: "032dd0c643627117f81a85304f3b63cf"
server_key_version: "encryptionkey@0"
I20260806 22:35:14.357677 30269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "f247277e4620467fa75f4bec06937bfa"
format_stamp: "Formatted at 2026-08-06 22:35:14 on dist-test-slave-h3ft"
server_key: "a9d197fc34b6d3bea8ab95c255e05e66"
server_key_iv: "032dd0c643627117f81a85304f3b63cf"
server_key_version: "encryptionkey@0"
I20260806 22:35:14.364965 30269 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260806 22:35:14.370049 30288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:14.372611 30269 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 22:35:14.372941 30269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "f247277e4620467fa75f4bec06937bfa"
format_stamp: "Formatted at 2026-08-06 22:35:14 on dist-test-slave-h3ft"
server_key: "a9d197fc34b6d3bea8ab95c255e05e66"
server_key_iv: "032dd0c643627117f81a85304f3b63cf"
server_key_version: "encryptionkey@0"
I20260806 22:35:14.373324 30269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:35:14.381771 30269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:35:14.383091 30269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:35:14.383533 30269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:35:14.408552 30269 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:33763
I20260806 22:35:14.408598 30358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:33763 every 8 connection(s)
I20260806 22:35:14.410524 30269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:35:14.415669 30359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:35:14.420974 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 30269
I20260806 22:35:14.421275 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:35:14.422016 16154 external_mini_cluster.cc:1428] Setting key 83fbbdd61e9cf9948281bfe87fca744c
I20260806 22:35:14.429807 30359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa: Bootstrap starting.
I20260806 22:35:14.432919 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:14.424740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48782 (local address 127.15.198.190:33763)
0806 22:35:14.426145 (+ 1405us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:14.426159 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:14.426220 (+ 61us) server_negotiation.cc:407] Connection header received
0806 22:35:14.428281 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:14.428313 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:14.428877 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:14.429378 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:14.430802 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:14.430810 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:14.430826 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:35:14.430841 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:14.430915 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:35:14.430934 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:14.431223 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:14.431387 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 22:35:14.431721 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":466,"threads_started":1}
I20260806 22:35:14.434249 30359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa: Neither blocks nor log segments found. Creating new log.
I20260806 22:35:14.435680 30359 log.cc:824] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa: Log is configured to *not* fsync() on all Append() calls
I20260806 22:35:14.439460 30359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa: No bootstrap required, opened a new log
I20260806 22:35:14.448398 30359 raft_consensus.cc:348] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f247277e4620467fa75f4bec06937bfa" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33763 } }
I20260806 22:35:14.449026 30359 raft_consensus.cc:374] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:35:14.449165 30359 raft_consensus.cc:729] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f247277e4620467fa75f4bec06937bfa, State: Initialized, Role: FOLLOWER
I20260806 22:35:14.449971 30359 consensus_queue.cc:260] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f247277e4620467fa75f4bec06937bfa" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33763 } }
I20260806 22:35:14.450275 30359 raft_consensus.cc:388] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:35:14.450441 30359 raft_consensus.cc:482] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:35:14.450680 30359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:35:14.453645 30359 raft_consensus.cc:504] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f247277e4620467fa75f4bec06937bfa" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33763 } }
I20260806 22:35:14.454314 30359 leader_election.cc:302] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f247277e4620467fa75f4bec06937bfa; no voters:
I20260806 22:35:14.455222 30359 leader_election.cc:288] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:35:14.455420 30368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:35:14.456693 30368 raft_consensus.cc:686] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [term 1 LEADER]: Becoming Leader. State: Replica: f247277e4620467fa75f4bec06937bfa, State: Running, Role: LEADER
I20260806 22:35:14.457381 30368 consensus_queue.cc:237] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f247277e4620467fa75f4bec06937bfa" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33763 } }
I20260806 22:35:14.458483 30359 sys_catalog.cc:564] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:35:14.461179 30369 sys_catalog.cc:455] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f247277e4620467fa75f4bec06937bfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f247277e4620467fa75f4bec06937bfa" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33763 } } }
I20260806 22:35:14.461989 30369 sys_catalog.cc:458] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [sys.catalog]: This master's current role is: LEADER
I20260806 22:35:14.461979 30370 sys_catalog.cc:455] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader f247277e4620467fa75f4bec06937bfa. Latest consensus state: current_term: 1 leader_uuid: "f247277e4620467fa75f4bec06937bfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f247277e4620467fa75f4bec06937bfa" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33763 } } }
I20260806 22:35:14.463436 30370 sys_catalog.cc:458] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa [sys.catalog]: This master's current role is: LEADER
I20260806 22:35:14.464802 30380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:35:14.471938 30380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:35:14.486061 30380 catalog_manager.cc:1347] Generated new cluster ID: fc183db5248d48769dfeb89971d9780c
I20260806 22:35:14.486243 30380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:35:14.502420 30380 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:35:14.503202 30380 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:35:14.518646 30380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f247277e4620467fa75f4bec06937bfa: Generated new TSK 0
I20260806 22:35:14.520061 30380 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:35:14.529247 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33763
--builtin_ntp_servers=127.15.198.148:41557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:35:14.995106 30392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:35:14.995527 30392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:35:14.995699 30392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:35:15.003979 30392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:35:15.004145 30392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:35:15.004341 30392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:35:15.015268 30392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:41557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:35:15.017175 30392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:35:15.019627 30392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:35:15.036648 30399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:15.036546 30402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:15.036831 30407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:35:15.037855 30392 server_base.cc:1034] running on GCE node
I20260806 22:35:15.038908 30392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:35:15.040606 30392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:35:15.041898 30392 hybrid_clock.cc:648] HybridClock initialized: now 1786055715041828 us; error 56 us; skew 500 ppm
I20260806 22:35:15.042397 30392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:35:15.045666 30392 webserver.cc:460] Webserver started at http://127.15.198.129:42309/ using document root <none> and password file <none>
I20260806 22:35:15.046697 30392 fs_manager.cc:359] Metadata directory not provided
I20260806 22:35:15.046839 30392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:35:15.047456 30392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:35:15.052102 30392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "89097255b4b74743bbe6c30b2210bca7"
format_stamp: "Formatted at 2026-08-06 22:35:15 on dist-test-slave-h3ft"
server_key: "dfaea4e07fa5e9df4bd4e1706c85e0f5"
server_key_iv: "0985a0310069dab79dcc9f5ef2fdf82a"
server_key_version: "encryptionkey@0"
I20260806 22:35:15.053198 30392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "89097255b4b74743bbe6c30b2210bca7"
format_stamp: "Formatted at 2026-08-06 22:35:15 on dist-test-slave-h3ft"
server_key: "dfaea4e07fa5e9df4bd4e1706c85e0f5"
server_key_iv: "0985a0310069dab79dcc9f5ef2fdf82a"
server_key_version: "encryptionkey@0"
I20260806 22:35:15.060536 30392 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 22:35:15.065896 30415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:15.067989 30392 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 22:35:15.068252 30392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "89097255b4b74743bbe6c30b2210bca7"
format_stamp: "Formatted at 2026-08-06 22:35:15 on dist-test-slave-h3ft"
server_key: "dfaea4e07fa5e9df4bd4e1706c85e0f5"
server_key_iv: "0985a0310069dab79dcc9f5ef2fdf82a"
server_key_version: "encryptionkey@0"
I20260806 22:35:15.068589 30392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:35:15.077196 30392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:35:15.078490 30392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:35:15.078835 30392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:35:15.080351 30392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:35:15.082329 30392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:35:15.082475 30392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:15.082639 30392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:35:15.082736 30392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:15.128221 30392 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:38991
I20260806 22:35:15.128269 30570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:38991 every 8 connection(s)
I20260806 22:35:15.130257 30392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:35:15.138598 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 30392
I20260806 22:35:15.138883 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:35:15.140002 16154 external_mini_cluster.cc:1428] Setting key f5848eca558fc3f561fecb5a46afcadf
I20260806 22:35:15.142953 30573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:15.134293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:42753 (local address 127.15.198.190:33763)
0806 22:35:15.134929 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:15.134936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:15.136897 (+ 1961us) server_negotiation.cc:407] Connection header received
0806 22:35:15.139680 (+ 2783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:15.139686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:15.139762 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:15.139888 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:15.141885 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:15.141892 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:15.141897 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:35:15.141925 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:15.141981 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:35:15.142000 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:15.142147 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:15.142439 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 22:35:15.142555 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":302,"threads_started":1}
I20260806 22:35:15.143925 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33763
--builtin_ntp_servers=127.15.198.148:41557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 22:35:15.144325 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:15.135338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33763 (local address 127.15.198.129:42753)
0806 22:35:15.136691 (+ 1353us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:15.136737 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:15.139153 (+ 2416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:15.140330 (+ 1177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:15.140346 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:15.140517 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:15.141505 (+ 988us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:35:15.141507 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:15.141558 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:35:15.141678 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:35:15.142125 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:35:15.142138 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:35:15.142147 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 22:35:15.142521 (+ 374us) client_negotiation.cc:240] Negotiation successful
0806 22:35:15.142859 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":362,"threads_started":1}
I20260806 22:35:15.146301 30571 heartbeater.cc:347] Connected to a master server at 127.15.198.190:33763
I20260806 22:35:15.146785 30571 heartbeater.cc:464] Registering TS with master...
I20260806 22:35:15.147773 30571 heartbeater.cc:511] Master 127.15.198.190:33763 requested a full tablet report, sending...
I20260806 22:35:15.150730 30311 ts_manager.cc:194] Registered new tserver with Master: 89097255b4b74743bbe6c30b2210bca7 (127.15.198.129:38991)
W20260806 22:35:15.590142 30577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:35:15.590597 30577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:35:15.590780 30577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:35:15.599223 30577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:35:15.599421 30577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:35:15.599630 30577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:35:15.610074 30577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:41557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:35:15.612042 30577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:35:15.614652 30577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:35:15.632781 30585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:15.632858 30584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:15.632741 30588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:35:15.634425 30577 server_base.cc:1034] running on GCE node
I20260806 22:35:15.635411 30577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:35:15.636811 30577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:35:15.638087 30577 hybrid_clock.cc:648] HybridClock initialized: now 1786055715637977 us; error 81 us; skew 500 ppm
I20260806 22:35:15.638551 30577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:35:15.649195 30577 webserver.cc:460] Webserver started at http://127.15.198.130:41465/ using document root <none> and password file <none>
I20260806 22:35:15.650388 30577 fs_manager.cc:359] Metadata directory not provided
I20260806 22:35:15.650558 30577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:35:15.651042 30577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:35:15.655877 30577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "77557c0f20704e2985cba65d07031d93"
format_stamp: "Formatted at 2026-08-06 22:35:15 on dist-test-slave-h3ft"
server_key: "1a8265e5ce4341bc5eeceee8862730c9"
server_key_iv: "b2254af49ae4a8af6ac89d2fce98e036"
server_key_version: "encryptionkey@0"
I20260806 22:35:15.657235 30577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "77557c0f20704e2985cba65d07031d93"
format_stamp: "Formatted at 2026-08-06 22:35:15 on dist-test-slave-h3ft"
server_key: "1a8265e5ce4341bc5eeceee8862730c9"
server_key_iv: "b2254af49ae4a8af6ac89d2fce98e036"
server_key_version: "encryptionkey@0"
I20260806 22:35:15.665043 30577 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 22:35:15.670311 30597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:15.672537 30577 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:35:15.672857 30577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "77557c0f20704e2985cba65d07031d93"
format_stamp: "Formatted at 2026-08-06 22:35:15 on dist-test-slave-h3ft"
server_key: "1a8265e5ce4341bc5eeceee8862730c9"
server_key_iv: "b2254af49ae4a8af6ac89d2fce98e036"
server_key_version: "encryptionkey@0"
I20260806 22:35:15.673187 30577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:35:15.681497 30577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:35:15.682705 30577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:35:15.683192 30577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:35:15.684624 30577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:35:15.686621 30577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:35:15.686753 30577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:15.686920 30577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:35:15.686995 30577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:15.732908 30577 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:44055
I20260806 22:35:15.732954 30754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:44055 every 8 connection(s)
I20260806 22:35:15.734748 30577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:35:15.738380 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 30577
I20260806 22:35:15.738646 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:35:15.739316 16154 external_mini_cluster.cc:1428] Setting key 30a84fcfe4696b9674c6c4c2ac0d1ae3
I20260806 22:35:15.743922 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33763
--builtin_ntp_servers=127.15.198.148:41557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 22:35:15.754027 30757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:15.738407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:39477 (local address 127.15.198.190:33763)
0806 22:35:15.739384 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:15.739418 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:15.741540 (+ 2122us) server_negotiation.cc:407] Connection header received
0806 22:35:15.744410 (+ 2870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:15.744420 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:15.744536 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:15.744679 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:15.752531 (+ 7852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:15.752542 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:15.752551 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:35:15.752616 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:15.752703 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:35:15.752734 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:15.752943 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:15.753214 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 22:35:15.753322 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":368,"threads_started":1}
I20260806 22:35:15.755510 30759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:15.739476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33763 (local address 127.15.198.130:39477)
0806 22:35:15.741211 (+ 1735us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:15.741294 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:15.743814 (+ 2520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:15.745398 (+ 1584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:15.745414 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:15.745584 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:15.751802 (+ 6218us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:35:15.751806 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:15.751888 (+ 82us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:35:15.752121 (+ 233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:35:15.752897 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:35:15.752909 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:35:15.752919 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 22:35:15.753308 (+ 389us) client_negotiation.cc:240] Negotiation successful
0806 22:35:15.753769 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":927,"thread_start_us":503,"threads_started":1}
I20260806 22:35:15.757792 30755 heartbeater.cc:347] Connected to a master server at 127.15.198.190:33763
I20260806 22:35:15.758275 30755 heartbeater.cc:464] Registering TS with master...
I20260806 22:35:15.759315 30755 heartbeater.cc:511] Master 127.15.198.190:33763 requested a full tablet report, sending...
I20260806 22:35:15.761636 30311 ts_manager.cc:194] Registered new tserver with Master: 77557c0f20704e2985cba65d07031d93 (127.15.198.130:44055)
I20260806 22:35:16.155462 30571 heartbeater.cc:503] Master 127.15.198.190:33763 was elected leader, sending a full tablet report...
W20260806 22:35:16.211338 30760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:35:16.211771 30760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:35:16.211941 30760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:35:16.220340 30760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:35:16.220497 30760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:35:16.220721 30760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:35:16.231154 30760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:41557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:35:16.233104 30760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:35:16.235496 30760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:35:16.252463 30770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:16.252447 30775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:16.252504 30772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:35:16.253643 30760 server_base.cc:1034] running on GCE node
I20260806 22:35:16.254384 30760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:35:16.255847 30760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:35:16.257122 30760 hybrid_clock.cc:648] HybridClock initialized: now 1786055716257070 us; error 41 us; skew 500 ppm
I20260806 22:35:16.257552 30760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:35:16.260927 30760 webserver.cc:460] Webserver started at http://127.15.198.131:43583/ using document root <none> and password file <none>
I20260806 22:35:16.261922 30760 fs_manager.cc:359] Metadata directory not provided
I20260806 22:35:16.262084 30760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:35:16.262504 30760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:35:16.266429 30760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "fbb80b8230184a13ae8f04306966763b"
format_stamp: "Formatted at 2026-08-06 22:35:16 on dist-test-slave-h3ft"
server_key: "46142df80156bd061625efbd18008ba2"
server_key_iv: "3d532a24e30456dc15e65493ed2dbaf6"
server_key_version: "encryptionkey@0"
I20260806 22:35:16.267488 30760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "fbb80b8230184a13ae8f04306966763b"
format_stamp: "Formatted at 2026-08-06 22:35:16 on dist-test-slave-h3ft"
server_key: "46142df80156bd061625efbd18008ba2"
server_key_iv: "3d532a24e30456dc15e65493ed2dbaf6"
server_key_version: "encryptionkey@0"
I20260806 22:35:16.274102 30760 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 22:35:16.279451 30782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:16.282243 30760 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260806 22:35:16.282581 30760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "fbb80b8230184a13ae8f04306966763b"
format_stamp: "Formatted at 2026-08-06 22:35:16 on dist-test-slave-h3ft"
server_key: "46142df80156bd061625efbd18008ba2"
server_key_iv: "3d532a24e30456dc15e65493ed2dbaf6"
server_key_version: "encryptionkey@0"
I20260806 22:35:16.282970 30760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:35:16.293756 30760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:35:16.294965 30760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:35:16.295400 30760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:35:16.296651 30760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:35:16.298684 30760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:35:16.298840 30760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:16.299039 30760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:35:16.299161 30760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260806 22:35:16.340874 30760 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:43493
I20260806 22:35:16.340999 30960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:43493 every 8 connection(s)
I20260806 22:35:16.342708 30760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:35:16.351670 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 30760
I20260806 22:35:16.351925 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:35:16.352481 16154 external_mini_cluster.cc:1428] Setting key 6c3e07d22b7c972c3c0fc597322aa188
I20260806 22:35:16.353819 30966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:16.346181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:38197 (local address 127.15.198.190:33763)
0806 22:35:16.346772 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:16.346780 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:16.348397 (+ 1617us) server_negotiation.cc:407] Connection header received
0806 22:35:16.350537 (+ 2140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:16.350542 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:16.350622 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:16.350754 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:16.352663 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:16.352671 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:16.352678 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:35:16.352711 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:16.352774 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:35:16.352788 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:16.352958 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:16.353233 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 22:35:16.353351 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":267,"threads_started":1}
I20260806 22:35:16.355496 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:16.346950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33763 (local address 127.15.198.131:38197)
0806 22:35:16.348165 (+ 1215us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:16.348214 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:16.350206 (+ 1992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:16.351151 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:16.351162 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:16.351282 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:16.352250 (+ 968us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:35:16.352253 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:16.352305 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:35:16.352419 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:35:16.352909 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:35:16.352921 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:35:16.352932 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 22:35:16.353338 (+ 406us) client_negotiation.cc:240] Negotiation successful
0806 22:35:16.353703 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":328,"threads_started":1}
I20260806 22:35:16.357173 30962 heartbeater.cc:347] Connected to a master server at 127.15.198.190:33763
I20260806 22:35:16.357517 30962 heartbeater.cc:464] Registering TS with master...
I20260806 22:35:16.358242 30962 heartbeater.cc:511] Master 127.15.198.190:33763 requested a full tablet report, sending...
I20260806 22:35:16.360050 30311 ts_manager.cc:194] Registered new tserver with Master: fbb80b8230184a13ae8f04306966763b (127.15.198.131:43493)
I20260806 22:35:16.369807 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:35:16.377880 30966 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:35:16.375801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48792 (local address 127.15.198.190:33763)
0806 22:35:16.376020 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:16.376025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:16.376650 (+ 625us) server_negotiation.cc:407] Connection header received
0806 22:35:16.376847 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:16.376851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:16.376924 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:16.377041 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:16.377560 (+ 519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48792: BlockingRecv error: recv got EOF from 127.0.0.1:48792 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 22:35:16.378631 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 30392
I20260806 22:35:16.383884 30977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:16.380472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48040 (local address 127.15.198.129:38991)
0806 22:35:16.381104 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:16.381124 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:16.381156 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:35:16.381234 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:16.381242 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:16.381527 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:16.381687 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:16.382916 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:16.382923 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:16.382938 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:35:16.382950 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:16.383059 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:35:16.383082 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:16.383253 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:16.383311 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 22:35:16.383440 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":363,"threads_started":1}
I20260806 22:35:16.385531 30563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:35:16.478747 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 30392
I20260806 22:35:16.511992 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 30577
I20260806 22:35:16.516641 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:16.513167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39102 (local address 127.15.198.130:44055)
0806 22:35:16.513847 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:16.513856 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:16.513893 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:35:16.513977 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:16.513985 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:16.514288 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:16.514463 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:16.515736 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:16.515744 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:16.515760 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:35:16.515774 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:16.515832 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:35:16.515851 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:16.516005 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:16.516175 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 22:35:16.516290 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":325,"threads_started":1}
I20260806 22:35:16.518617 30749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:35:16.609596 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 30577
I20260806 22:35:16.641464 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 30760
I20260806 22:35:16.646807 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:16.642887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43264 (local address 127.15.198.131:43493)
0806 22:35:16.643835 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:16.643843 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:16.643885 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:35:16.643971 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:16.643979 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:16.644283 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:16.644443 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:16.645871 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:16.645882 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:16.645900 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:35:16.645927 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:16.645999 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:35:16.646026 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:16.646186 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:16.646251 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 22:35:16.646384 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":429,"threads_started":1}
I20260806 22:35:16.648737 30953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:35:16.743286 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 30760
I20260806 22:35:16.773135 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 30269
I20260806 22:35:16.774828 30353 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:35:16.900214 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 30269
2026-08-06T22:35:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (3166 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:35:17 dist-test-slave-h3ft krb5kdc[30992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:35:17 dist-test-slave-h3ft krb5kdc[30992](info): set up 2 sockets
Aug 06 22:35:17 dist-test-slave-h3ft krb5kdc[30992](info): commencing operation
krb5kdc: starting...
W20260806 22:35:18.299741 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.339s user 0.000s sys 0.060s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:35:18 dist-test-slave-h3ft krb5kdc[30992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:35:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:35:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:35:18.704553 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:42105
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:42065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:42105
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:35:19.170608 31013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:35:19.171096 31013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:35:19.171284 31013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:35:19.179947 31013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:35:19.180119 31013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:35:19.180214 31013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:35:19.180330 31013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:35:19.180423 31013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:35:19.192293 31013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:42065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:42105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:42105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:35:19.194272 31013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:35:19.196153 31013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:35:19.211652 31020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:19.211640 31024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:19.211843 31021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:35:19.212745 31013 server_base.cc:1034] running on GCE node
I20260806 22:35:19.213812 31013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:35:19.215878 31013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:35:19.217170 31013 hybrid_clock.cc:648] HybridClock initialized: now 1786055719217106 us; error 44 us; skew 500 ppm
I20260806 22:35:19.217579 31013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:35:19 dist-test-slave-h3ft krb5kdc[30992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055719, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:35:19.221789 31013 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:35:19.223891 31013 webserver.cc:460] Webserver started at http://127.15.198.190:46259/ using document root <none> and password file <none>
I20260806 22:35:19.224821 31013 fs_manager.cc:359] Metadata directory not provided
I20260806 22:35:19.224974 31013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:35:19.225399 31013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:35:19.230142 31013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "9e585915f1b545edaaaec7c0b4dc1ce6"
format_stamp: "Formatted at 2026-08-06 22:35:19 on dist-test-slave-h3ft"
server_key: "4dd60a9b96e8d6df9c7e2250de043694"
server_key_iv: "cfd7bfcfe11a6e48d0ae0173c4e77033"
server_key_version: "encryptionkey@0"
I20260806 22:35:19.231292 31013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "9e585915f1b545edaaaec7c0b4dc1ce6"
format_stamp: "Formatted at 2026-08-06 22:35:19 on dist-test-slave-h3ft"
server_key: "4dd60a9b96e8d6df9c7e2250de043694"
server_key_iv: "cfd7bfcfe11a6e48d0ae0173c4e77033"
server_key_version: "encryptionkey@0"
I20260806 22:35:19.238359 31013 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260806 22:35:19.242985 31034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:19.245227 31013 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:35:19.245432 31013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "9e585915f1b545edaaaec7c0b4dc1ce6"
format_stamp: "Formatted at 2026-08-06 22:35:19 on dist-test-slave-h3ft"
server_key: "4dd60a9b96e8d6df9c7e2250de043694"
server_key_iv: "cfd7bfcfe11a6e48d0ae0173c4e77033"
server_key_version: "encryptionkey@0"
I20260806 22:35:19.245672 31013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:35:19.299693 31013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:35:19.305449 31013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:35:19.305831 31013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:35:19.329041 31013 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:42105
I20260806 22:35:19.329077 31099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:42105 every 8 connection(s)
I20260806 22:35:19.330901 31013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:35:19.333276 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 31013
I20260806 22:35:19.333585 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:35:19.334532 16154 external_mini_cluster.cc:1428] Setting key 67fc20b1bcc2fcf5b654087af42e1cbe
I20260806 22:35:19.337221 31100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 22:35:19 dist-test-slave-h3ft krb5kdc[30992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:35:19.354758 31100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6: Bootstrap starting.
I20260806 22:35:19.359205 31100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6: Neither blocks nor log segments found. Creating new log.
I20260806 22:35:19.360837 31100 log.cc:824] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:35:19.364110 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:19.337211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47762 (local address 127.15.198.190:42105)
0806 22:35:19.338216 (+ 1005us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:19.338231 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:19.338294 (+ 63us) server_negotiation.cc:407] Connection header received
0806 22:35:19.340296 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:19.340326 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:19.340856 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:19.341354 (+ 498us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:19.343706 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:19.345338 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:19.347381 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:19.348315 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:19.348492 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:19.354519 (+ 6027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:19.354530 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:19.354555 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:35:19.354617 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:19.358270 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:35:19.359463 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:35:19.359472 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:35:19.359480 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:35:19.359766 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:35:19.360260 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:35:19.360264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:35:19.360266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:35:19.361050 (+ 784us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:19.361340 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:19.361984 (+ 644us) server_negotiation.cc:299] Negotiation successful
0806 22:35:19.362599 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":453,"threads_started":1}
I20260806 22:35:19.365170 31100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6: No bootstrap required, opened a new log
I20260806 22:35:19.374118 31100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e585915f1b545edaaaec7c0b4dc1ce6" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42105 } }
I20260806 22:35:19.374724 31100 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:35:19.374853 31100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e585915f1b545edaaaec7c0b4dc1ce6, State: Initialized, Role: FOLLOWER
I20260806 22:35:19.375674 31100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e585915f1b545edaaaec7c0b4dc1ce6" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42105 } }
I20260806 22:35:19.375967 31100 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:35:19.376148 31100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:35:19.376366 31100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:35:19.379480 31100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e585915f1b545edaaaec7c0b4dc1ce6" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42105 } }
I20260806 22:35:19.380209 31100 leader_election.cc:302] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e585915f1b545edaaaec7c0b4dc1ce6; no voters:
I20260806 22:35:19.380995 31100 leader_election.cc:288] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:35:19.381183 31105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:35:19.382082 31105 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [term 1 LEADER]: Becoming Leader. State: Replica: 9e585915f1b545edaaaec7c0b4dc1ce6, State: Running, Role: LEADER
I20260806 22:35:19.382727 31105 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e585915f1b545edaaaec7c0b4dc1ce6" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42105 } }
I20260806 22:35:19.383769 31100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:35:19.386754 31110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e585915f1b545edaaaec7c0b4dc1ce6. Latest consensus state: current_term: 1 leader_uuid: "9e585915f1b545edaaaec7c0b4dc1ce6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e585915f1b545edaaaec7c0b4dc1ce6" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42105 } } }
I20260806 22:35:19.386840 31107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e585915f1b545edaaaec7c0b4dc1ce6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e585915f1b545edaaaec7c0b4dc1ce6" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42105 } } }
I20260806 22:35:19.387274 31110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:35:19.387419 31107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:35:19.388475 31117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:35:19.396064 31117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:35:19.408041 31117 catalog_manager.cc:1347] Generated new cluster ID: f114a533c5994b3b8d992244b7e465a7
I20260806 22:35:19.408281 31117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:35:19.425012 31117 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:35:19.427461 31117 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:35:19.436044 31117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6: Generated new TSK 0
I20260806 22:35:19.437403 31117 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
I20260806 22:35:19.543974 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42105
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:42065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:35:19.994280 31137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:35:19.994685 31137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:35:19.994793 31137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:35:20.003453 31137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:35:20.003659 31137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:35:20.003799 31137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:35:20.016497 31137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:42065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:35:20.018463 31137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:35:20.020364 31137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:35:20.038856 31144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:20.038867 31147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:20.038758 31145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:35:20.040024 31137 server_base.cc:1034] running on GCE node
I20260806 22:35:20.040891 31137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:35:20.042446 31137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:35:20.043939 31137 hybrid_clock.cc:648] HybridClock initialized: now 1786055720043854 us; error 71 us; skew 500 ppm
I20260806 22:35:20.044339 31137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:35:20 dist-test-slave-h3ft krb5kdc[30992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055720, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:35:20.048708 31137 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:35:20.050823 31137 webserver.cc:460] Webserver started at http://127.15.198.129:45987/ using document root <none> and password file <none>
I20260806 22:35:20.051925 31137 fs_manager.cc:359] Metadata directory not provided
I20260806 22:35:20.052062 31137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:35:20.052407 31137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:35:20.056452 31137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16"
format_stamp: "Formatted at 2026-08-06 22:35:20 on dist-test-slave-h3ft"
server_key: "e791c45754c984635ddc66edfb4cec0f"
server_key_iv: "6e4222d9196b68cbd919a33253c4360d"
server_key_version: "encryptionkey@0"
I20260806 22:35:20.057463 31137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16"
format_stamp: "Formatted at 2026-08-06 22:35:20 on dist-test-slave-h3ft"
server_key: "e791c45754c984635ddc66edfb4cec0f"
server_key_iv: "6e4222d9196b68cbd919a33253c4360d"
server_key_version: "encryptionkey@0"
I20260806 22:35:20.065840 31137 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260806 22:35:20.071733 31157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:20.074293 31137 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260806 22:35:20.074533 31137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16"
format_stamp: "Formatted at 2026-08-06 22:35:20 on dist-test-slave-h3ft"
server_key: "e791c45754c984635ddc66edfb4cec0f"
server_key_iv: "6e4222d9196b68cbd919a33253c4360d"
server_key_version: "encryptionkey@0"
I20260806 22:35:20.074823 31137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:35:20.108026 31137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:35:20.113844 31137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:35:20.114207 31137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:35:20.115494 31137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:35:20.117342 31137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:35:20.117431 31137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:20.117535 31137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:35:20.117583 31137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:20.158601 31137 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:46493
I20260806 22:35:20.158653 31332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:46493 every 8 connection(s)
I20260806 22:35:20.160455 31137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:35:20.168818 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 31137
I20260806 22:35:20.169108 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:35:20.169826 16154 external_mini_cluster.cc:1428] Setting key cdbbee7d7ee3ae4977f64cc7d166c625
Aug 06 22:35:20 dist-test-slave-h3ft krb5kdc[30992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055720, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:35:20.187512 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:20.163943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:41173 (local address 127.15.198.190:42105)
0806 22:35:20.164563 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:20.164569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:20.165704 (+ 1135us) server_negotiation.cc:407] Connection header received
0806 22:35:20.168033 (+ 2329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:20.168040 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:20.168156 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:20.168312 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:20.170540 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:20.171550 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:20.173702 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:20.174497 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:20.174660 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:20.179134 (+ 4474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:20.179143 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:20.179151 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:35:20.179208 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:20.183232 (+ 4024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:35:20.184198 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:35:20.184206 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:35:20.184212 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:35:20.184402 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:35:20.184960 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:35:20.184975 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:35:20.184980 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:35:20.185468 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:20.185642 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:20.186620 (+ 978us) server_negotiation.cc:299] Negotiation successful
0806 22:35:20.186936 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":327,"threads_started":1}
I20260806 22:35:20.188658 31336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:20.164548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:42105 (local address 127.15.198.129:41173)
0806 22:35:20.165513 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:20.165562 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:20.167659 (+ 2097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:20.168609 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:20.168620 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:20.169456 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:20.170315 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:20.170338 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:20.171759 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:20.171764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:20.173516 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:20.173537 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:20.174769 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:20.174775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:20.175084 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:20.175946 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:20.175973 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:20.178796 (+ 2823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:35:20.183399 (+ 4603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:35:20.183408 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:35:20.183426 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:35:20.184072 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:35:20.184550 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:35:20.184554 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:35:20.184556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:35:20.184842 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:35:20.185645 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:35:20.185653 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:35:20.186144 (+ 491us) client_negotiation.cc:769] Sending connection context
0806 22:35:20.186472 (+ 328us) client_negotiation.cc:240] Negotiation successful
0806 22:35:20.187083 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":324,"threads_started":1}
I20260806 22:35:20.190838 31333 heartbeater.cc:347] Connected to a master server at 127.15.198.190:42105
I20260806 22:35:20.191391 31333 heartbeater.cc:464] Registering TS with master...
I20260806 22:35:20.192602 31333 heartbeater.cc:511] Master 127.15.198.190:42105 requested a full tablet report, sending...
I20260806 22:35:20.195556 31059 ts_manager.cc:194] Registered new tserver with Master: c3aef6a4f7564a02b0d8f1feb4d3ec16 (127.15.198.129:46493)
I20260806 22:35:20.197419 31059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:41173
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
I20260806 22:35:20.256603 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42105
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:42065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:35:20.706960 31340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:35:20.707585 31340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:35:20.707713 31340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:35:20.716460 31340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:35:20.716658 31340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:35:20.716799 31340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:35:20.728874 31340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:42065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:35:20.731223 31340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:35:20.733101 31340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:35:20.750615 31348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:20.750659 31349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:20.750710 31353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:35:20.752182 31340 server_base.cc:1034] running on GCE node
I20260806 22:35:20.753062 31340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:35:20.754416 31340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:35:20.755678 31340 hybrid_clock.cc:648] HybridClock initialized: now 1786055720755623 us; error 52 us; skew 500 ppm
I20260806 22:35:20.756145 31340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:35:20 dist-test-slave-h3ft krb5kdc[30992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055720, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:35:20.765167 31340 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:35:20.767589 31340 webserver.cc:460] Webserver started at http://127.15.198.130:42413/ using document root <none> and password file <none>
I20260806 22:35:20.768556 31340 fs_manager.cc:359] Metadata directory not provided
I20260806 22:35:20.768734 31340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:35:20.769130 31340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:35:20.773064 31340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "b627c5bf954e410aa1880014e73dc87c"
format_stamp: "Formatted at 2026-08-06 22:35:20 on dist-test-slave-h3ft"
server_key: "d6d6df24fe3f7c26863c16a65ae480e8"
server_key_iv: "a0f05035c0de3f76bfec5e326066f513"
server_key_version: "encryptionkey@0"
I20260806 22:35:20.774096 31340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "b627c5bf954e410aa1880014e73dc87c"
format_stamp: "Formatted at 2026-08-06 22:35:20 on dist-test-slave-h3ft"
server_key: "d6d6df24fe3f7c26863c16a65ae480e8"
server_key_iv: "a0f05035c0de3f76bfec5e326066f513"
server_key_version: "encryptionkey@0"
I20260806 22:35:20.781265 31340 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.003s
I20260806 22:35:20.786314 31361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:20.788379 31340 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:35:20.788647 31340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "b627c5bf954e410aa1880014e73dc87c"
format_stamp: "Formatted at 2026-08-06 22:35:20 on dist-test-slave-h3ft"
server_key: "d6d6df24fe3f7c26863c16a65ae480e8"
server_key_iv: "a0f05035c0de3f76bfec5e326066f513"
server_key_version: "encryptionkey@0"
I20260806 22:35:20.788925 31340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:35:20.803875 31340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:35:20.809510 31340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:35:20.809938 31340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:35:20.811327 31340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:35:20.813326 31340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:35:20.813457 31340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:20.813638 31340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:35:20.813724 31340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:20.858466 31340 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:39973
I20260806 22:35:20.858500 31528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:39973 every 8 connection(s)
I20260806 22:35:20.860328 31340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:35:20.861581 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 31340
I20260806 22:35:20.861853 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:35:20.862457 16154 external_mini_cluster.cc:1428] Setting key fcfcf50ed415560cac163c8c70ceaac2
Aug 06 22:35:20 dist-test-slave-h3ft krb5kdc[30992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055720, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:35:20.891855 31531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:20.863901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:59623 (local address 127.15.198.190:42105)
0806 22:35:20.864552 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:20.864560 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:20.866007 (+ 1447us) server_negotiation.cc:407] Connection header received
0806 22:35:20.869763 (+ 3756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:20.869768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:20.869861 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:20.869977 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:20.873036 (+ 3059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:20.874022 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:20.877182 (+ 3160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:20.877714 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:20.877834 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:20.884081 (+ 6247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:20.884088 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:20.884113 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:35:20.884196 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:20.887589 (+ 3393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:35:20.888663 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:35:20.888668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:35:20.888671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:35:20.888870 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:35:20.889426 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:35:20.889431 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:35:20.889435 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:35:20.889881 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:20.890074 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:20.891067 (+ 993us) server_negotiation.cc:299] Negotiation successful
0806 22:35:20.891385 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":316,"threads_started":1}
I20260806 22:35:20.893388 31535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:20.864725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:42105 (local address 127.15.198.130:59623)
0806 22:35:20.865761 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:20.865848 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:20.869254 (+ 3406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:20.870372 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:20.870389 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:20.871505 (+ 1116us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:20.872676 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:20.872717 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:20.874255 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:20.874264 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:20.876920 (+ 2656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:20.876957 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:20.878033 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:20.878044 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:20.878485 (+ 441us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:20.879630 (+ 1145us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:20.879676 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:20.883662 (+ 3986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:35:20.887802 (+ 4140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:35:20.887814 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:35:20.887845 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:35:20.888438 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:35:20.889008 (+ 570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:35:20.889013 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:35:20.889018 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:35:20.889279 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:35:20.890032 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:35:20.890043 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:35:20.890567 (+ 524us) client_negotiation.cc:769] Sending connection context
0806 22:35:20.890905 (+ 338us) client_negotiation.cc:240] Negotiation successful
0806 22:35:20.891721 (+ 816us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":392,"threads_started":1}
I20260806 22:35:20.895771 31529 heartbeater.cc:347] Connected to a master server at 127.15.198.190:42105
I20260806 22:35:20.896245 31529 heartbeater.cc:464] Registering TS with master...
I20260806 22:35:20.897266 31529 heartbeater.cc:511] Master 127.15.198.190:42105 requested a full tablet report, sending...
I20260806 22:35:20.899601 31059 ts_manager.cc:194] Registered new tserver with Master: b627c5bf954e410aa1880014e73dc87c (127.15.198.130:39973)
I20260806 22:35:20.901117 31059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:59623
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
I20260806 22:35:20.952267 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42105
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:42065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:35:21.201583 31333 heartbeater.cc:503] Master 127.15.198.190:42105 was elected leader, sending a full tablet report...
W20260806 22:35:21.400566 31538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:35:21.400990 31538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:35:21.401121 31538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:35:21.409495 31538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:35:21.409677 31538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:35:21.409814 31538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:35:21.421438 31538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:42065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:42105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:35:21.423370 31538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:35:21.425069 31538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:35:21.442368 31544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:21.442394 31548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:35:21.442870 31545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:35:21.443760 31538 server_base.cc:1034] running on GCE node
I20260806 22:35:21.444604 31538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:35:21.445869 31538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:35:21.447161 31538 hybrid_clock.cc:648] HybridClock initialized: now 1786055721447069 us; error 79 us; skew 500 ppm
I20260806 22:35:21.447602 31538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:35:21 dist-test-slave-h3ft krb5kdc[30992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055721, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:35:21.452672 31538 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:35:21.454872 31538 webserver.cc:460] Webserver started at http://127.15.198.131:37027/ using document root <none> and password file <none>
I20260806 22:35:21.456013 31538 fs_manager.cc:359] Metadata directory not provided
I20260806 22:35:21.456220 31538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:35:21.456665 31538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:35:21.461344 31538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "5834d6ddf0ee437a9c6a5ea45f65c77c"
format_stamp: "Formatted at 2026-08-06 22:35:21 on dist-test-slave-h3ft"
server_key: "604a63daedcdae01bb41a4cb34f70a7a"
server_key_iv: "3337974131827be412fd3e1ddd659b07"
server_key_version: "encryptionkey@0"
I20260806 22:35:21.462541 31538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "5834d6ddf0ee437a9c6a5ea45f65c77c"
format_stamp: "Formatted at 2026-08-06 22:35:21 on dist-test-slave-h3ft"
server_key: "604a63daedcdae01bb41a4cb34f70a7a"
server_key_iv: "3337974131827be412fd3e1ddd659b07"
server_key_version: "encryptionkey@0"
I20260806 22:35:21.470122 31538 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 22:35:21.475211 31559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:21.477250 31538 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:35:21.477521 31538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "5834d6ddf0ee437a9c6a5ea45f65c77c"
format_stamp: "Formatted at 2026-08-06 22:35:21 on dist-test-slave-h3ft"
server_key: "604a63daedcdae01bb41a4cb34f70a7a"
server_key_iv: "3337974131827be412fd3e1ddd659b07"
server_key_version: "encryptionkey@0"
I20260806 22:35:21.477810 31538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:35:21.504536 31538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:35:21.511726 31538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:35:21.512212 31538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:35:21.513697 31538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:35:21.515998 31538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:35:21.516114 31538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:21.516268 31538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:35:21.516336 31538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:35:21.559105 31538 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:43903
I20260806 22:35:21.559139 31722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:43903 every 8 connection(s)
I20260806 22:35:21.560935 31538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:35:21.567147 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 31538
I20260806 22:35:21.567425 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:35:21.568081 16154 external_mini_cluster.cc:1428] Setting key 4a6049f0c7e7842b916b8ee11edd2050
Aug 06 22:35:21 dist-test-slave-h3ft krb5kdc[30992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055721, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:35:21.588534 31725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:21.564590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:40105 (local address 127.15.198.190:42105)
0806 22:35:21.565235 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:21.565240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:21.566377 (+ 1137us) server_negotiation.cc:407] Connection header received
0806 22:35:21.569701 (+ 3324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:21.569708 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:21.569848 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:21.569995 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:21.572649 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.573541 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.575451 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.575925 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.576082 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:21.580696 (+ 4614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:21.580702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:21.580706 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:35:21.580753 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:21.584400 (+ 3647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:35:21.585727 (+ 1327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:35:21.585733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:35:21.585736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:35:21.585903 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:35:21.586410 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:35:21.586414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:35:21.586417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:35:21.586759 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:21.586894 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:21.587898 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0806 22:35:21.588108 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":322,"threads_started":1}
I20260806 22:35:21.590018 31726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:21.565237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:42105 (local address 127.15.198.131:40105)
0806 22:35:21.566162 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:21.566212 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:21.569237 (+ 3025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:21.570474 (+ 1237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:21.570490 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:21.571483 (+ 993us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:35:21.572395 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:21.572443 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.573718 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.573722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:21.575291 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:21.575307 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.576203 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.576208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:21.576465 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:21.577299 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:35:21.577331 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:35:21.580283 (+ 2952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:35:21.584623 (+ 4340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:35:21.584633 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:35:21.584652 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:35:21.585580 (+ 928us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:35:21.586049 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:35:21.586053 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:35:21.586056 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:35:21.586285 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:35:21.586918 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:35:21.586927 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:35:21.587481 (+ 554us) client_negotiation.cc:769] Sending connection context
0806 22:35:21.587778 (+ 297us) client_negotiation.cc:240] Negotiation successful
0806 22:35:21.588425 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":331,"threads_started":1}
I20260806 22:35:21.591882 31723 heartbeater.cc:347] Connected to a master server at 127.15.198.190:42105
I20260806 22:35:21.592305 31723 heartbeater.cc:464] Registering TS with master...
I20260806 22:35:21.593449 31723 heartbeater.cc:511] Master 127.15.198.190:42105 requested a full tablet report, sending...
I20260806 22:35:21.595502 31059 ts_manager.cc:194] Registered new tserver with Master: 5834d6ddf0ee437a9c6a5ea45f65c77c (127.15.198.131:43903)
I20260806 22:35:21.596524 31059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:40105
I20260806 22:35:21.598837 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:35:21.619767 31725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:21.604458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47776 (local address 127.15.198.190:42105)
0806 22:35:21.604671 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:21.604676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:21.605218 (+ 542us) server_negotiation.cc:407] Connection header received
0806 22:35:21.605469 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:21.605475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:21.605605 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:21.605738 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:21.607545 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.608384 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.609788 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.610253 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.610361 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:21.613364 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:21.613380 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:21.613384 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:35:21.613427 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:21.616640 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:35:21.617552 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:35:21.617557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:35:21.617559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:35:21.617709 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:35:21.618081 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:35:21.618085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:35:21.618087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:35:21.618504 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:21.618719 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:21.619248 (+ 529us) server_negotiation.cc:299] Negotiation successful
0806 22:35:21.619476 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 22:35:21.625391 31059 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:35:21.634192 31059 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:35:21.664372 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:21.651236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:39973 (local address 127.0.0.1:48590)
0806 22:35:21.653591 (+ 2355us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:21.653650 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:21.653925 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:21.655471 (+ 1546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:21.655497 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:21.655564 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:35:21.656122 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:21.656141 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.658434 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.658440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:21.661346 (+ 2906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:21.661360 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.663098 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.663107 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:21.663345 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:21.663418 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 22:35:21.663580 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 22:35:21.663836 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2033,"mutex_wait_us":2,"thread_start_us":1842,"threads_started":1}
I20260806 22:35:21.664803 31740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:21.651663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:43903 (local address 127.0.0.1:49554)
0806 22:35:21.654614 (+ 2951us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:21.654632 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:21.654773 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:21.656068 (+ 1295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:21.656077 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:21.656119 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:35:21.656770 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:21.656786 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.658913 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.658917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:21.661391 (+ 2474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:21.661410 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.663725 (+ 2315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.663731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:21.663935 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:21.663973 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:35:21.664061 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:35:21.664227 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2784,"mutex_wait_us":55,"thread_start_us":2591,"threads_started":1}
I20260806 22:35:21.666059 31748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:21.654713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:46493 (local address 127.0.0.1:46948)
0806 22:35:21.656903 (+ 2190us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:21.656921 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:21.657053 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:21.657700 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:21.657705 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:21.657729 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:35:21.658147 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:21.658161 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.660765 (+ 2604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.660770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:21.663133 (+ 2363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:21.663150 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.665237 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.665243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:21.665417 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:21.665453 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:35:21.665550 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 22:35:21.665694 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2034,"thread_start_us":1849,"threads_started":1}
I20260806 22:35:21.666092 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:21.651634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48590 (local address 127.15.198.130:39973)
0806 22:35:21.652492 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:21.652504 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:21.654219 (+ 1715us) server_negotiation.cc:407] Connection header received
0806 22:35:21.654356 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:21.654368 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:21.654720 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:21.655090 (+ 370us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:35:21.656361 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.658270 (+ 1909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.661543 (+ 3273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.662759 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.665061 (+ 2302us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:21.665337 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:21.665427 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 22:35:21.665696 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":476,"threads_started":1}
I20260806 22:35:21.666158 31750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:21.655347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46948 (local address 127.15.198.129:46493)
0806 22:35:21.656100 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:21.656110 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:21.657056 (+ 946us) server_negotiation.cc:407] Connection header received
0806 22:35:21.657207 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:21.657219 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:21.657468 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:21.657710 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:35:21.658411 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.660587 (+ 2176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.663379 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.664994 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.665230 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:21.665408 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:21.665616 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 22:35:21.665783 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":345,"threads_started":1}
I20260806 22:35:21.666618 31746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:21.652642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49554 (local address 127.15.198.131:43903)
0806 22:35:21.653555 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:21.653565 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:21.654943 (+ 1378us) server_negotiation.cc:407] Connection header received
0806 22:35:21.655198 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:21.655208 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:21.655443 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:21.655666 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:35:21.657007 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.658736 (+ 1729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.661646 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.663336 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.665271 (+ 1935us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:21.665462 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:21.665560 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 22:35:21.665759 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":494,"threads_started":1}
I20260806 22:35:21.671418 31233 tablet_service.cc:1467] Processing CreateTablet for tablet 5d92664530c04f0699c5e8676055bbd7 (DEFAULT_TABLE table=test-table [id=976bb1f14cd6476f9fcf9ab99b65ee5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:35:21.671432 31434 tablet_service.cc:1467] Processing CreateTablet for tablet 5d92664530c04f0699c5e8676055bbd7 (DEFAULT_TABLE table=test-table [id=976bb1f14cd6476f9fcf9ab99b65ee5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:35:21.671435 31634 tablet_service.cc:1467] Processing CreateTablet for tablet 5d92664530c04f0699c5e8676055bbd7 (DEFAULT_TABLE table=test-table [id=976bb1f14cd6476f9fcf9ab99b65ee5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:35:21.673173 31233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d92664530c04f0699c5e8676055bbd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:35:21.673207 31634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d92664530c04f0699c5e8676055bbd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:35:21.673219 31434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d92664530c04f0699c5e8676055bbd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:35:21.689455 31751 tablet_bootstrap.cc:492] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16: Bootstrap starting.
I20260806 22:35:21.692616 31754 tablet_bootstrap.cc:492] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c: Bootstrap starting.
I20260806 22:35:21.694012 31753 tablet_bootstrap.cc:492] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c: Bootstrap starting.
I20260806 22:35:21.697377 31751 tablet_bootstrap.cc:654] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16: Neither blocks nor log segments found. Creating new log.
I20260806 22:35:21.697468 31754 tablet_bootstrap.cc:654] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c: Neither blocks nor log segments found. Creating new log.
I20260806 22:35:21.698216 31753 tablet_bootstrap.cc:654] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c: Neither blocks nor log segments found. Creating new log.
I20260806 22:35:21.698874 31754 log.cc:824] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:35:21.698994 31751 log.cc:824] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16: Log is configured to *not* fsync() on all Append() calls
I20260806 22:35:21.700465 31753 log.cc:824] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:35:21.703872 31754 tablet_bootstrap.cc:492] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c: No bootstrap required, opened a new log
I20260806 22:35:21.704161 31751 tablet_bootstrap.cc:492] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16: No bootstrap required, opened a new log
I20260806 22:35:21.704299 31754 ts_tablet_manager.cc:1397] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260806 22:35:21.704473 31751 ts_tablet_manager.cc:1397] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.001s
I20260806 22:35:21.704990 31753 tablet_bootstrap.cc:492] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c: No bootstrap required, opened a new log
I20260806 22:35:21.705807 31753 ts_tablet_manager.cc:1397] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260806 22:35:21.714560 31753 raft_consensus.cc:348] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b627c5bf954e410aa1880014e73dc87c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39973 } } peers { permanent_uuid: "5834d6ddf0ee437a9c6a5ea45f65c77c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43903 } } peers { permanent_uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46493 } }
I20260806 22:35:21.715188 31753 raft_consensus.cc:374] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:35:21.715303 31753 raft_consensus.cc:729] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b627c5bf954e410aa1880014e73dc87c, State: Initialized, Role: FOLLOWER
I20260806 22:35:21.716209 31753 consensus_queue.cc:260] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b627c5bf954e410aa1880014e73dc87c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39973 } } peers { permanent_uuid: "5834d6ddf0ee437a9c6a5ea45f65c77c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43903 } } peers { permanent_uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46493 } }
I20260806 22:35:21.717103 31751 raft_consensus.cc:348] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b627c5bf954e410aa1880014e73dc87c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39973 } } peers { permanent_uuid: "5834d6ddf0ee437a9c6a5ea45f65c77c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43903 } } peers { permanent_uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46493 } }
I20260806 22:35:21.717103 31754 raft_consensus.cc:348] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b627c5bf954e410aa1880014e73dc87c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39973 } } peers { permanent_uuid: "5834d6ddf0ee437a9c6a5ea45f65c77c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43903 } } peers { permanent_uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46493 } }
I20260806 22:35:21.717733 31754 raft_consensus.cc:374] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:35:21.717733 31751 raft_consensus.cc:374] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:35:21.717777 31529 heartbeater.cc:503] Master 127.15.198.190:42105 was elected leader, sending a full tablet report...
I20260806 22:35:21.717849 31751 raft_consensus.cc:729] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3aef6a4f7564a02b0d8f1feb4d3ec16, State: Initialized, Role: FOLLOWER
I20260806 22:35:21.717850 31754 raft_consensus.cc:729] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5834d6ddf0ee437a9c6a5ea45f65c77c, State: Initialized, Role: FOLLOWER
I20260806 22:35:21.718712 31751 consensus_queue.cc:260] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b627c5bf954e410aa1880014e73dc87c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39973 } } peers { permanent_uuid: "5834d6ddf0ee437a9c6a5ea45f65c77c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43903 } } peers { permanent_uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46493 } }
I20260806 22:35:21.718955 31753 ts_tablet_manager.cc:1428] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
I20260806 22:35:21.718713 31754 consensus_queue.cc:260] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b627c5bf954e410aa1880014e73dc87c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39973 } } peers { permanent_uuid: "5834d6ddf0ee437a9c6a5ea45f65c77c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43903 } } peers { permanent_uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46493 } }
I20260806 22:35:21.723816 31754 ts_tablet_manager.cc:1428] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c: Time spent starting tablet: real 0.019s user 0.020s sys 0.000s
I20260806 22:35:21.724645 31723 heartbeater.cc:503] Master 127.15.198.190:42105 was elected leader, sending a full tablet report...
I20260806 22:35:21.729326 31764 raft_consensus.cc:482] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:35:21.729734 31764 raft_consensus.cc:504] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b627c5bf954e410aa1880014e73dc87c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39973 } } peers { permanent_uuid: "5834d6ddf0ee437a9c6a5ea45f65c77c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43903 } } peers { permanent_uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46493 } }
I20260806 22:35:21.730214 31751 ts_tablet_manager.cc:1428] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16: Time spent starting tablet: real 0.026s user 0.012s sys 0.014s
I20260806 22:35:21.732771 31764 leader_election.cc:288] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b627c5bf954e410aa1880014e73dc87c (127.15.198.130:39973), 5834d6ddf0ee437a9c6a5ea45f65c77c (127.15.198.131:43903)
I20260806 22:35:21.743539 31746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:21.733457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:44413 (local address 127.15.198.131:43903)
0806 22:35:21.733705 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:21.733713 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:21.735628 (+ 1915us) server_negotiation.cc:407] Connection header received
0806 22:35:21.735870 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:21.735876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:21.735990 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:21.736064 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:35:21.737720 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.738893 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.741291 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.742208 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.742466 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:21.742583 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:21.742954 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 22:35:21.743171 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 22:35:21.743536 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:21.733463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:51267 (local address 127.15.198.130:39973)
0806 22:35:21.733931 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:21.733940 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:21.735301 (+ 1361us) server_negotiation.cc:407] Connection header received
0806 22:35:21.735777 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:21.735787 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:21.735891 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:21.735990 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:35:21.737893 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.739094 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.741290 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.742210 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.742439 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:21.742555 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:21.742984 (+ 429us) server_negotiation.cc:299] Negotiation successful
0806 22:35:21.743181 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280}
I20260806 22:35:21.743721 31767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:21.734192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:43903 (local address 127.15.198.129:44413)
0806 22:35:21.735482 (+ 1290us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:21.735511 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:21.735681 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:21.736273 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:21.736280 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:21.736305 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:35:21.737485 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:21.737498 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.739106 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.739113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:21.741048 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:21.741069 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.742510 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.742518 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:21.742745 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:21.742789 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:35:21.742878 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 22:35:21.743178 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1142,"mutex_wait_us":489,"thread_start_us":951,"threads_started":1}
I20260806 22:35:21.744534 31766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:21.733497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:39973 (local address 127.15.198.129:51267)
0806 22:35:21.735117 (+ 1620us) negotiation.cc:107] Waiting for socket to connect
0806 22:35:21.735157 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:35:21.735482 (+ 325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:35:21.736216 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:35:21.736223 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:35:21.736254 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:35:21.737261 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:21.737281 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.739240 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.739245 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:21.741059 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:35:21.741074 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.742563 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.742571 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:35:21.742760 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:21.742804 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:35:21.742923 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:35:21.743180 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1378,"mutex_wait_us":156,"thread_start_us":1134,"threads_started":1}
I20260806 22:35:21.745194 31658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d92664530c04f0699c5e8676055bbd7" candidate_uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5834d6ddf0ee437a9c6a5ea45f65c77c" is_pre_election: true
I20260806 22:35:21.745805 31463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d92664530c04f0699c5e8676055bbd7" candidate_uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b627c5bf954e410aa1880014e73dc87c" is_pre_election: true
I20260806 22:35:21.745913 31658 raft_consensus.cc:2454] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3aef6a4f7564a02b0d8f1feb4d3ec16 in term 0.
I20260806 22:35:21.746389 31463 raft_consensus.cc:2454] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3aef6a4f7564a02b0d8f1feb4d3ec16 in term 0.
I20260806 22:35:21.746912 31164 leader_election.cc:302] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5834d6ddf0ee437a9c6a5ea45f65c77c, c3aef6a4f7564a02b0d8f1feb4d3ec16; no voters:
I20260806 22:35:21.747530 31764 raft_consensus.cc:2781] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:35:21.747720 31764 raft_consensus.cc:482] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:35:21.747853 31764 raft_consensus.cc:3037] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:35:21.750723 31764 raft_consensus.cc:504] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b627c5bf954e410aa1880014e73dc87c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39973 } } peers { permanent_uuid: "5834d6ddf0ee437a9c6a5ea45f65c77c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43903 } } peers { permanent_uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46493 } }
I20260806 22:35:21.751909 31764 leader_election.cc:288] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [CANDIDATE]: Term 1 election: Requested vote from peers b627c5bf954e410aa1880014e73dc87c (127.15.198.130:39973), 5834d6ddf0ee437a9c6a5ea45f65c77c (127.15.198.131:43903)
I20260806 22:35:21.752804 31463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d92664530c04f0699c5e8676055bbd7" candidate_uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b627c5bf954e410aa1880014e73dc87c"
I20260806 22:35:21.752811 31658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d92664530c04f0699c5e8676055bbd7" candidate_uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5834d6ddf0ee437a9c6a5ea45f65c77c"
I20260806 22:35:21.753155 31658 raft_consensus.cc:3037] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:35:21.753149 31463 raft_consensus.cc:3037] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:35:21.757289 31463 raft_consensus.cc:2454] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3aef6a4f7564a02b0d8f1feb4d3ec16 in term 1.
I20260806 22:35:21.757289 31658 raft_consensus.cc:2454] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3aef6a4f7564a02b0d8f1feb4d3ec16 in term 1.
I20260806 22:35:21.758002 31164 leader_election.cc:302] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5834d6ddf0ee437a9c6a5ea45f65c77c, c3aef6a4f7564a02b0d8f1feb4d3ec16; no voters:
I20260806 22:35:21.758791 31764 raft_consensus.cc:2781] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:35:21.759042 31764 raft_consensus.cc:686] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [term 1 LEADER]: Becoming Leader. State: Replica: c3aef6a4f7564a02b0d8f1feb4d3ec16, State: Running, Role: LEADER
I20260806 22:35:21.759754 31764 consensus_queue.cc:237] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b627c5bf954e410aa1880014e73dc87c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39973 } } peers { permanent_uuid: "5834d6ddf0ee437a9c6a5ea45f65c77c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43903 } } peers { permanent_uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46493 } }
I20260806 22:35:21.765453 31058 catalog_manager.cc:5495] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3aef6a4f7564a02b0d8f1feb4d3ec16 (127.15.198.129). New cstate: current_term: 1 leader_uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b627c5bf954e410aa1880014e73dc87c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5834d6ddf0ee437a9c6a5ea45f65c77c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3aef6a4f7564a02b0d8f1feb4d3ec16" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 46493 } health_report { overall_health: HEALTHY } } }
I20260806 22:35:21.802520 31750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:21.795780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46952 (local address 127.15.198.129:46493)
0806 22:35:21.796098 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:21.796103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:21.796123 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:35:21.796257 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:21.796261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:21.796370 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:21.796586 (+ 216us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:35:21.797105 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.797958 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.799678 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.800165 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.800338 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:21.801000 (+ 662us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:35:21.801735 (+ 735us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:35:21.801929 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:21.802041 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 22:35:21.802157 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
W20260806 22:35:21.813057 31724 tablet.cc:2369] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:35:21.815109 31658 raft_consensus.cc:1261] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c [term 1 FOLLOWER]: Refusing update from remote peer c3aef6a4f7564a02b0d8f1feb4d3ec16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:35:21.815330 31463 raft_consensus.cc:1261] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c [term 1 FOLLOWER]: Refusing update from remote peer c3aef6a4f7564a02b0d8f1feb4d3ec16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:35:21.816680 31764 consensus_queue.cc:1035] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5834d6ddf0ee437a9c6a5ea45f65c77c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 43903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:35:21.817204 31761 consensus_queue.cc:1035] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b627c5bf954e410aa1880014e73dc87c" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:35:21.851346 31058 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:35:21.851816 31058 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:35:21.856840 31058 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e585915f1b545edaaaec7c0b4dc1ce6: Sending DeleteTablet for 3 replicas of tablet 5d92664530c04f0699c5e8676055bbd7
I20260806 22:35:21.858944 31434 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d92664530c04f0699c5e8676055bbd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:35:21 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:48590
I20260806 22:35:21.862229 31233 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d92664530c04f0699c5e8676055bbd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:35:21 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:46948
I20260806 22:35:21.862300 31634 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d92664530c04f0699c5e8676055bbd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:35:21 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:49554
I20260806 22:35:21.865206 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 31137
I20260806 22:35:21.885284 31791 tablet_replica.cc:330] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16: stopping tablet replica
I20260806 22:35:21.887650 31791 raft_consensus.cc:2229] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:35:21.888767 31792 tablet_replica.cc:330] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c: stopping tablet replica
I20260806 22:35:21.889547 31792 raft_consensus.cc:2229] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:35:21.890374 31792 raft_consensus.cc:2258] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:35:21.893761 31792 ts_tablet_manager.cc:1905] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:35:21.896145 31791 raft_consensus.cc:2258] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:35:21.900285 31791 ts_tablet_manager.cc:1905] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:35:21.907768 31790 tablet_replica.cc:330] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c: stopping tablet replica
I20260806 22:35:21.908381 31790 raft_consensus.cc:2229] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:35:21.909052 31790 raft_consensus.cc:2258] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 06 22:35:21 dist-test-slave-h3ft krb5kdc[30992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:35:21.912267 31790 ts_tablet_manager.cc:1905] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:35:21.914458 31792 ts_tablet_manager.cc:1918] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:35:21.914701 31792 log.cc:1196] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/5d92664530c04f0699c5e8676055bbd7
I20260806 22:35:21.916922 31792 ts_tablet_manager.cc:1939] T 5d92664530c04f0699c5e8676055bbd7 P 5834d6ddf0ee437a9c6a5ea45f65c77c: Deleting consensus metadata
I20260806 22:35:21.920488 31042 catalog_manager.cc:4841] TS 5834d6ddf0ee437a9c6a5ea45f65c77c (127.15.198.131:43903): tablet 5d92664530c04f0699c5e8676055bbd7 (table test-table [id=976bb1f14cd6476f9fcf9ab99b65ee5a]) successfully deleted
I20260806 22:35:21.919236 31791 ts_tablet_manager.cc:1918] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:35:21.921258 31791 log.cc:1196] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/5d92664530c04f0699c5e8676055bbd7
I20260806 22:35:21.921972 31791 ts_tablet_manager.cc:1939] T 5d92664530c04f0699c5e8676055bbd7 P c3aef6a4f7564a02b0d8f1feb4d3ec16: Deleting consensus metadata
I20260806 22:35:21.927137 31750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:21.867801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46966 (local address 127.15.198.129:46493)
0806 22:35:21.868096 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:21.868106 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:21.882351 (+ 14245us) server_negotiation.cc:407] Connection header received
0806 22:35:21.884023 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:21.884031 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:21.884172 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:21.884264 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:21.895624 (+ 11360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.897142 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.901244 (+ 4102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:21.902030 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:21.902164 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:21.916659 (+ 14495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:21.916689 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:21.916711 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:35:21.916781 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:21.922161 (+ 5380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:35:21.923479 (+ 1318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:35:21.923492 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:35:21.923496 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:35:21.923749 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:35:21.924312 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:35:21.924317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:35:21.924321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:35:21.924915 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:21.925287 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:21.926284 (+ 997us) server_negotiation.cc:299] Negotiation successful
0806 22:35:21.926622 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 22:35:21.928079 31327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:35:21.928898 31790 ts_tablet_manager.cc:1918] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:35:21.929255 31790 log.cc:1196] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/5d92664530c04f0699c5e8676055bbd7
I20260806 22:35:21.930218 31790 ts_tablet_manager.cc:1939] T 5d92664530c04f0699c5e8676055bbd7 P b627c5bf954e410aa1880014e73dc87c: Deleting consensus metadata
I20260806 22:35:21.932754 31038 catalog_manager.cc:4841] TS b627c5bf954e410aa1880014e73dc87c (127.15.198.130:39973): tablet 5d92664530c04f0699c5e8676055bbd7 (table test-table [id=976bb1f14cd6476f9fcf9ab99b65ee5a]) successfully deleted
I20260806 22:35:22.044179 31042 catalog_manager.cc:4841] TS c3aef6a4f7564a02b0d8f1feb4d3ec16 (127.15.198.129:46493): tablet 5d92664530c04f0699c5e8676055bbd7 (table test-table [id=976bb1f14cd6476f9fcf9ab99b65ee5a]) successfully deleted
I20260806 22:35:22.046296 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 31137
I20260806 22:35:22.080361 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 31340
Aug 06 22:35:22 dist-test-slave-h3ft krb5kdc[30992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.130@KRBTEST.COM
I20260806 22:35:22.100355 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:22.081349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48596 (local address 127.15.198.130:39973)
0806 22:35:22.081554 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:22.081559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:22.081582 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:35:22.081825 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:22.081830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:22.081917 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:22.082015 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:22.083726 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:22.084701 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:22.086238 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:22.086680 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:22.086820 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:22.092799 (+ 5979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:22.092823 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:22.092847 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:35:22.092900 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:22.096666 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:35:22.097619 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:35:22.097629 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:35:22.097632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:35:22.097803 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:35:22.098260 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:35:22.098266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:35:22.098272 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:35:22.098846 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:22.099262 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:22.099659 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 22:35:22.099942 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 22:35:22.101308 31521 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:35:22.203203 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 31340
I20260806 22:35:22.235925 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 31538
Aug 06 22:35:22 dist-test-slave-h3ft krb5kdc[30992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.131@KRBTEST.COM
I20260806 22:35:22.255254 31746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:35:22.237139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49556 (local address 127.15.198.131:43903)
0806 22:35:22.237362 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 22:35:22.237367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:35:22.237391 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:35:22.237663 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:35:22.237668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:35:22.237773 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:35:22.237881 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:35:22.239455 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:22.240292 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:22.241577 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:35:22.242026 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:35:22.242132 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:35:22.247544 (+ 5412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:35:22.247567 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:35:22.247591 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:35:22.247643 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:35:22.251554 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:35:22.252545 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:35:22.252555 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:35:22.252558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:35:22.252776 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:35:22.253228 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:35:22.253232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:35:22.253234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:35:22.253735 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:35:22.254096 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0806 22:35:22.254457 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 22:35:22.254691 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 22:35:22.256138 31718 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:35:22.357460 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 31538
I20260806 22:35:22.390352 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 31013
I20260806 22:35:22.391996 31095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:35:22.525323 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 31013
2026-08-06T22:35:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (5625 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:35:22 dist-test-slave-h3ft krb5kdc[31804](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:35:22 dist-test-slave-h3ft krb5kdc[31804](info): set up 2 sockets
Aug 06 22:35:22 dist-test-slave-h3ft krb5kdc[31804](info): commencing operation
krb5kdc: starting...
W20260806 22:35:23.931541 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.346s user 0.000s sys 0.061s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:35:24 dist-test-slave-h3ft krb5kdc[31804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:35:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:35:24Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.198.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.198.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.198.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/rangeradmin_127.15.198.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.198.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/rangeradmin_127.15.198.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.198.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.198.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.198.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/rangerlookup_127.15.198.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.198.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/rangerlookup_127.15.198.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.198.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.198.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.198.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/HTTP_127.15.198.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.198.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/HTTP_127.15.198.148@KRBTEST.COM.keytab.
I20260806 22:35:24.512290 16154 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskuRfOoI/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/postgres -l logfile start
2026-08-06 22:35:27.674 UTC [31841] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 22:35:27.675 UTC [31841] LOG: listening on IPv4 address "127.15.198.148", port 58703
2026-08-06 22:35:27.680 UTC [31841] LOG: listening on Unix socket "/tmp/.s.PGSQL.58703"
2026-08-06 22:35:27.713 UTC [31844] LOG: database system was shut down at 2026-08-06 22:35:25 UTC
2026-08-06 22:35:27.718 UTC [31841] LOG: database system is ready to accept connections
I20260806 22:35:28.933020 16154 mini_postgres.cc:96] Postgres bound to 58703
2026-08-06 22:35:28.971 UTC [31858] FATAL: database "slave" does not exist
127.15.198.148:58703 - accepting connections
I20260806 22:35:28.972535 16154 mini_ranger.cc:161] Starting Ranger...
I20260806 22:35:29.019524 16154 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 22:35:29.661837 16154 mini_ranger.cc:87] Created ranger Postgres database
I20260806 22:35:29.661991 16154 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ranger-admin
2026-08-06 22:35:30,043 [I] DB FLAVOR :POSTGRES
2026-08-06 22:35:30,044 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:35:30,044 [I] Checking connection..
2026-08-06 22:35:30,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 22:35:30,361 [I] Checking connection passed.
2026-08-06 22:35:30,362 [I] --------- Verifying version history table ---------
2026-08-06 22:35:30,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:35:30,659 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:35:30,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:35:30,962 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:35:30,962 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 22:35:30,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 22:35:31.258 UTC [31948] WARNING: there is no transaction in progress
2026-08-06 22:35:31,270 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 22:35:31,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:35:31,590 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 22:35:31,591 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:35:31,591 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 22:35:31,884 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 22:35:32,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h3ft.c.gcp-upstream.internal','N') ;"
2026-08-06 22:35:32,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:35:32,766 [I] Table x_portal_user does not exist in database ranger
2026-08-06 22:35:32,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:35:33,047 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 22:35:33,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:35:33,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:35:33,636 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 22:35:33,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 22:35:34.841 UTC [32098] WARNING: there is no transaction in progress
2026-08-06 22:35:34.865 UTC [32098] WARNING: there is no transaction in progress
2026-08-06 22:35:34.887 UTC [32098] WARNING: there is no transaction in progress
2026-08-06 22:35:34.907 UTC [32098] WARNING: there is no transaction in progress
2026-08-06 22:35:34.928 UTC [32098] WARNING: there is no transaction in progress
2026-08-06 22:35:34.955 UTC [32098] WARNING: there is no transaction in progress
2026-08-06 22:35:35.033 UTC [32098] WARNING: there is no transaction in progress
2026-08-06 22:35:35.042 UTC [32098] WARNING: there is no transaction in progress
2026-08-06 22:35:35.059 UTC [32098] WARNING: there is no transaction in progress
2026-08-06 22:35:35.072 UTC [32098] WARNING: there is no transaction in progress
2026-08-06 22:35:35.085 UTC [32098] WARNING: there is no transaction in progress
2026-08-06 22:35:35.105 UTC [32098] WARNING: there is no transaction in progress
2026-08-06 22:35:35.117 UTC [32098] WARNING: there is no transaction in progress
2026-08-06 22:35:35.126 UTC [32098] WARNING: there is no transaction in progress
2026-08-06 22:35:35.136 UTC [32098] WARNING: there is no transaction in progress
2026-08-06 22:35:35.145 UTC [32098] WARNING: there is no transaction in progress
2026-08-06 22:35:36,016 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 22:35:36,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 22:35:36,333 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 22:35:36,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:35:36,644 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 22:35:36,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:35:36,958 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 22:35:36,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:35:37,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:35:37,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h3ft.c.gcp-upstream.internal';"
2026-08-06 22:35:37,817 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 22:35:37,818 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 22:35:38,113 [I] DB_PATCHES have already been applied
I20260806 22:35:38.120771 16154 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ranger-admin:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskuRfOoI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 22:35:38.120927 16154 mini_ranger.cc:193] Using host: 127.15.198.148
I20260806 22:35:38.150564 16154 mini_ranger.cc:227] Ranger admin URL: http://127.15.198.148:33899
Aug 06, 2026 10:35:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:35:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:35:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 10:35:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 10:35:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 10:35:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 10:35:39 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 22:35:39.589658 16154 mini_ranger.cc:160] Time spent starting Ranger: real 10.617s user 0.004s sys 0.093s
Aug 06, 2026 10:36:03 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 10:36:03 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 10:36:03 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 10:36:04 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 10:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 10:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 10:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 10:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 10:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 10:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 10:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 10:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 10:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 10:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 10:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 10:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 10:36:04 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 10:36:05 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 22:36:06.196395 16154 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.198.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.198.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.198.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/rangerkms_127.15.198.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.198.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/rangerkms_127.15.198.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.198.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.198.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.198.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/HTTP_127.15.198.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.198.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/HTTP_127.15.198.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 22:36:06 dist-test-slave-h3ft krb5kdc[31804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055766, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 22:36:06.521486 16154 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 22:36:06.564551 16154 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 22:36:07.633566 16154 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 22:36:07.633808 16154 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ranger-kms
I20260806 22:36:07.634002 16154 mini_ranger_kms.cc:227] Using postgres at 127.15.198.148:58703
2026-08-06 22:36:08,343 [I] DB FLAVOR :POSTGRES
2026-08-06 22:36:08,344 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:36:08,344 [I] Checking connection
2026-08-06 22:36:08,344 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 22:36:08,640 [I] connection success
2026-08-06 22:36:08,640 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 22:36:08,641 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 22:36:08,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 22:36:08,942 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 22:36:08,942 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:36:08,943 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 22:36:08,943 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:58703/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 22:36:09,263 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 22:36:09.903803 16154 mini_ranger_kms.cc:310] Created kms service
I20260806 22:36:10.146044 16154 mini_ranger_kms.cc:326] Created kudu user
I20260806 22:36:10.201212 16154 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 22:36:10 dist-test-slave-h3ft krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055766, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
I20260806 22:36:10.704548 16154 mini_ranger_kms.cc:384] Added ranger policy
I20260806 22:36:10.705073 16154 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ranger-kms:/tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 22:36:10.705171 16154 mini_ranger_kms.cc:240] Using host: 127.15.198.148
I20260806 22:36:10.747398 16154 mini_ranger_kms.cc:276] Ranger KMS PID: 32388
I20260806 22:36:10.747668 16154 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.198.148:33733
Aug 06, 2026 10:36:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:36:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:36:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 10:36:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 10:36:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 10:36:12 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 22:36:13 dist-test-slave-h3ft krb5kdc[31804](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786055773, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.198.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:36:13.707373 16154 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 7.186s user 0.200s sys 0.585s
I20260806 22:36:13.707814 16154 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 22:36:13.708097 16154 mini_ranger_kms.cc:401] 127.15.198.148:33733/kms/v1/keys
Aug 06 22:36:13 dist-test-slave-h3ft krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055773, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.198.148@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
Aug 06 22:36:14 dist-test-slave-h3ft krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055773, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.198.148@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
Aug 06 22:36:17 dist-test-slave-h3ft krb5kdc[31804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
W20260806 22:36:17.227157 16154 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.198.190: real 0.107s user 0.005s sys 0.040s
I20260806 22:36:17.228818 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:40181
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:34371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:40181
--ranger_config_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.198.148:33733/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:36:17.788326 32463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:36:17.788898 32463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:36:17.789201 32463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:36:17.800601 32463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:36:17.800879 32463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:36:17.800997 32463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:36:17.801190 32463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:36:17.801319 32463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:36:17.815392 32463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:34371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.198.148:33733/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:40181
--ranger_config_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:40181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:36:17.817341 32463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:36:17.819293 32463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:36:17.836365 32475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:36:17.836416 32477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:36:17.836390 32479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:36:17.838057 32463 server_base.cc:1034] running on GCE node
I20260806 22:36:17.839284 32463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:36:17.841234 32463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:36:17.842489 32463 hybrid_clock.cc:648] HybridClock initialized: now 1786055777842438 us; error 43 us; skew 500 ppm
I20260806 22:36:17.842945 32463 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:36:17 dist-test-slave-h3ft krb5kdc[31804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055777, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:36:17.853327 32463 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:36:17.855703 32463 webserver.cc:460] Webserver started at http://127.15.198.190:46659/ using document root <none> and password file <none>
I20260806 22:36:17.856592 32463 fs_manager.cc:359] Metadata directory not provided
I20260806 22:36:17.856760 32463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:36:17.857173 32463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:36:17 dist-test-slave-h3ft krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055777, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
I20260806 22:36:17.928761 32463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "3a7713413d76402ba4f8ea2a016683b7"
format_stamp: "Formatted at 2026-08-06 22:36:17 on dist-test-slave-h3ft"
server_key: "2f89e8dd30075e2a096ae707cfbb5183"
server_key_iv: "7ca3903ec96d97e4947992c2da9e5742"
server_key_version: "kuduclusterkey@0"
I20260806 22:36:17.929953 32463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "3a7713413d76402ba4f8ea2a016683b7"
format_stamp: "Formatted at 2026-08-06 22:36:17 on dist-test-slave-h3ft"
server_key: "2f89e8dd30075e2a096ae707cfbb5183"
server_key_iv: "7ca3903ec96d97e4947992c2da9e5742"
server_key_version: "kuduclusterkey@0"
I20260806 22:36:17.937529 32463 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 22:36:17.965122 32488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:36:17.967871 32463 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260806 22:36:17.968187 32463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "3a7713413d76402ba4f8ea2a016683b7"
format_stamp: "Formatted at 2026-08-06 22:36:17 on dist-test-slave-h3ft"
server_key: "2f89e8dd30075e2a096ae707cfbb5183"
server_key_iv: "7ca3903ec96d97e4947992c2da9e5742"
server_key_version: "kuduclusterkey@0"
I20260806 22:36:17.968573 32463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:36:18.004335 32463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:36:18.010197 32463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:36:18.010579 32463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:36:18.039556 32463 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:40181
I20260806 22:36:18.039582 32562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:40181 every 8 connection(s)
I20260806 22:36:18.041513 32463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:36:18.047287 32565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:36:18.048002 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 32463
I20260806 22:36:18.048245 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:36:18.049080 16154 external_mini_cluster.cc:1428] Setting key 05a3c2f71a2d74002340cd2de5917ba9
I20260806 22:36:18.065377 32565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7: Bootstrap starting.
Aug 06 22:36:18 dist-test-slave-h3ft krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:36:18.072974 32565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7: Neither blocks nor log segments found. Creating new log.
I20260806 22:36:18.076679 32565 log.cc:824] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:36:18.082648 32565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7: No bootstrap required, opened a new log
I20260806 22:36:18.084110 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:18.052253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40312 (local address 127.15.198.190:40181)
0806 22:36:18.053763 (+ 1510us) server_negotiation.cc:206] Beginning negotiation
0806 22:36:18.053779 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:36:18.053853 (+ 74us) server_negotiation.cc:407] Connection header received
0806 22:36:18.056172 (+ 2319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:36:18.056215 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:36:18.056868 (+ 653us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:36:18.057448 (+ 580us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:36:18.059380 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:18.060981 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:18.062494 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:18.063625 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:18.063879 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:18.069939 (+ 6060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:36:18.069955 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:36:18.069978 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:36:18.070037 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:36:18.076720 (+ 6683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:36:18.078219 (+ 1499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:36:18.078249 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:36:18.078259 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:36:18.078642 (+ 383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:36:18.079770 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:36:18.079777 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:36:18.079783 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:36:18.080649 (+ 866us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:36:18.080968 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 22:36:18.081571 (+ 603us) server_negotiation.cc:299] Negotiation successful
0806 22:36:18.082026 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1130,"thread_start_us":784,"threads_started":1}
I20260806 22:36:18.097138 32565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a7713413d76402ba4f8ea2a016683b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40181 } }
I20260806 22:36:18.097781 32565 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:36:18.097919 32565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a7713413d76402ba4f8ea2a016683b7, State: Initialized, Role: FOLLOWER
I20260806 22:36:18.098830 32565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a7713413d76402ba4f8ea2a016683b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40181 } }
I20260806 22:36:18.099232 32565 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:36:18.099416 32565 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:36:18.099677 32565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:36:18.104377 32565 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a7713413d76402ba4f8ea2a016683b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40181 } }
I20260806 22:36:18.105278 32565 leader_election.cc:302] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a7713413d76402ba4f8ea2a016683b7; no voters:
I20260806 22:36:18.106784 32565 leader_election.cc:288] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:36:18.107033 32571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:36:18.110086 32571 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [term 1 LEADER]: Becoming Leader. State: Replica: 3a7713413d76402ba4f8ea2a016683b7, State: Running, Role: LEADER
I20260806 22:36:18.110862 32565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:36:18.110954 32571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a7713413d76402ba4f8ea2a016683b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40181 } }
I20260806 22:36:18.118328 32574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a7713413d76402ba4f8ea2a016683b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a7713413d76402ba4f8ea2a016683b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40181 } } }
I20260806 22:36:18.118358 32576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a7713413d76402ba4f8ea2a016683b7. Latest consensus state: current_term: 1 leader_uuid: "3a7713413d76402ba4f8ea2a016683b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a7713413d76402ba4f8ea2a016683b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40181 } } }
I20260806 22:36:18.119776 32576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:36:18.119606 32574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:36:18.126101 32587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:36:18.134950 32587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:36:18.135978 32565 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 22:36:18.169878 32587 catalog_manager.cc:1347] Generated new cluster ID: 3ac774681fdf4a83a75e11e29716d31d
I20260806 22:36:18.170166 32587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:36:18.217133 32587 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:36:18.220147 32587 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:36:18.235675 32587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7: Generated new TSK 0
I20260806 22:36:18.237457 32587 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
I20260806 22:36:19.228732 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.198.148:33733/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:40181
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:34371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:36:19 dist-test-slave-h3ft krb5kdc[31804](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786055779, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 22:36:19.885926 32632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:36:19.886611 32632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:36:19.886857 32632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:36:19.902029 32632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:36:19.902338 32632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:36:19.902612 32632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:36:19.922294 32632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:34371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.198.148:33733/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:40181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:36:19.925144 32632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:36:19.927800 32632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:36:19.964430 32653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:36:19.969105 32655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:36:19.972134 32632 server_base.cc:1034] running on GCE node
W20260806 22:36:19.973244 32657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:36:19.974750 32632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:36:19.977386 32632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:36:19.978886 32632 hybrid_clock.cc:648] HybridClock initialized: now 1786055779978633 us; error 131 us; skew 500 ppm
I20260806 22:36:19.979609 32632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:36:19 dist-test-slave-h3ft krb5kdc[31804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055779, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:36:19.987596 32632 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:36:19.990803 32632 webserver.cc:460] Webserver started at http://127.15.198.129:35007/ using document root <none> and password file <none>
I20260806 22:36:19.992159 32632 fs_manager.cc:359] Metadata directory not provided
I20260806 22:36:19.992449 32632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:36:19.993037 32632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:36:19 dist-test-slave-h3ft krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055779, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
I20260806 22:36:20.024025 32632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "4cdf1c42e5924d01a10a839644cf3ba1"
format_stamp: "Formatted at 2026-08-06 22:36:20 on dist-test-slave-h3ft"
server_key: "13bb9bda3c0ef5d75f4a1b7a4fa1d0fb"
server_key_iv: "cca8ce0e885392aacb5fdfc51c7e03c2"
server_key_version: "kuduclusterkey@0"
I20260806 22:36:20.026140 32632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "4cdf1c42e5924d01a10a839644cf3ba1"
format_stamp: "Formatted at 2026-08-06 22:36:20 on dist-test-slave-h3ft"
server_key: "13bb9bda3c0ef5d75f4a1b7a4fa1d0fb"
server_key_iv: "cca8ce0e885392aacb5fdfc51c7e03c2"
server_key_version: "kuduclusterkey@0"
I20260806 22:36:20.049932 32632 fs_manager.cc:691] Time spent creating directory manager: real 0.023s user 0.007s sys 0.011s
Aug 06 22:36:20 dist-test-slave-h3ft krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055779, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
I20260806 22:36:20.096119 32666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:36:20.099531 32632 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.008s sys 0.006s
I20260806 22:36:20.100056 32632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "4cdf1c42e5924d01a10a839644cf3ba1"
format_stamp: "Formatted at 2026-08-06 22:36:20 on dist-test-slave-h3ft"
server_key: "13bb9bda3c0ef5d75f4a1b7a4fa1d0fb"
server_key_iv: "cca8ce0e885392aacb5fdfc51c7e03c2"
server_key_version: "kuduclusterkey@0"
I20260806 22:36:20.100651 32632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:36:20.138294 32632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:36:20.149164 32632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:36:20.149945 32632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:36:20.164961 32632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:36:20.167865 32632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:36:20.168037 32632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:36:20.168254 32632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:36:20.168354 32632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:36:20.249972 32632 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:41995
I20260806 22:36:20.250106 364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:41995 every 8 connection(s)
I20260806 22:36:20.252183 32632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:36:20.262303 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 32632
I20260806 22:36:20.262611 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:36:20.263619 16154 external_mini_cluster.cc:1428] Setting key 3991b1f01624dffd75603150658bfad1
Aug 06 22:36:20 dist-test-slave-h3ft krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055779, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:36:20.288650 372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:20.256844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:57457 (local address 127.15.198.190:40181)
0806 22:36:20.257767 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0806 22:36:20.257774 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:36:20.260030 (+ 2256us) server_negotiation.cc:407] Connection header received
0806 22:36:20.263501 (+ 3471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:36:20.263509 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:36:20.263630 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:36:20.263727 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:36:20.266670 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:20.268025 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:20.271781 (+ 3756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:20.272500 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:20.272636 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:20.279165 (+ 6529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:36:20.279175 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:36:20.279180 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:36:20.279236 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:36:20.283583 (+ 4347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:36:20.284974 (+ 1391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:36:20.284983 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:36:20.284987 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:36:20.285209 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:36:20.285924 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:36:20.285930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:36:20.285933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:36:20.286394 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:36:20.286546 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 22:36:20.287816 (+ 1270us) server_negotiation.cc:299] Negotiation successful
0806 22:36:20.288150 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":465,"threads_started":1}
I20260806 22:36:20.291826 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:20.258162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:40181 (local address 127.15.198.129:57457)
0806 22:36:20.259784 (+ 1622us) negotiation.cc:107] Waiting for socket to connect
0806 22:36:20.259856 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 22:36:20.262978 (+ 3122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:36:20.264476 (+ 1498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:36:20.264492 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:36:20.265328 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:36:20.266426 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:36:20.266458 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:36:20.268291 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:36:20.268296 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:36:20.271525 (+ 3229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:36:20.271560 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:36:20.273105 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:36:20.273113 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:36:20.273653 (+ 540us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:20.274923 (+ 1270us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:36:20.275040 (+ 117us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:36:20.278686 (+ 3646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:36:20.283923 (+ 5237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:36:20.283937 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:36:20.283972 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:36:20.284777 (+ 805us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:36:20.285401 (+ 624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:36:20.285407 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:36:20.285410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:36:20.285772 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:36:20.286653 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:36:20.286664 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:36:20.287290 (+ 626us) client_negotiation.cc:769] Sending connection context
0806 22:36:20.288990 (+ 1700us) client_negotiation.cc:240] Negotiation successful
0806 22:36:20.289773 (+ 783us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1204,"thread_start_us":786,"threads_started":1}
I20260806 22:36:20.294973 369 heartbeater.cc:347] Connected to a master server at 127.15.198.190:40181
I20260806 22:36:20.295470 369 heartbeater.cc:464] Registering TS with master...
I20260806 22:36:20.296728 369 heartbeater.cc:511] Master 127.15.198.190:40181 requested a full tablet report, sending...
I20260806 22:36:20.300513 32512 ts_manager.cc:194] Registered new tserver with Master: 4cdf1c42e5924d01a10a839644cf3ba1 (127.15.198.129:41995)
I20260806 22:36:20.303426 32512 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:57457
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
W20260806 22:36:20.374956 16154 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.198.130: real 0.107s user 0.001s sys 0.032s
I20260806 22:36:20.376117 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.198.148:33733/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:40181
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:34371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:36:20 dist-test-slave-h3ft krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055779, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
W20260806 22:36:21.040407 376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:36:21.040940 376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:36:21.041141 376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:36:21.050532 376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:36:21.050760 376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:36:21.050972 376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:36:21.063441 376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:34371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.198.148:33733/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:40181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:36:21.065524 376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:36:21.067509 376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:36:21.088338 395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:36:21.088402 396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:36:21.089460 400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:36:21.090559 376 server_base.cc:1034] running on GCE node
I20260806 22:36:21.091951 376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:36:21.093472 376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:36:21.094761 376 hybrid_clock.cc:648] HybridClock initialized: now 1786055781094688 us; error 58 us; skew 500 ppm
I20260806 22:36:21.095249 376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:36:21 dist-test-slave-h3ft krb5kdc[31804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055781, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:36:21.100225 376 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:36:21.102524 376 webserver.cc:460] Webserver started at http://127.15.198.130:46195/ using document root <none> and password file <none>
I20260806 22:36:21.103617 376 fs_manager.cc:359] Metadata directory not provided
I20260806 22:36:21.103790 376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:36:21.104188 376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:36:21 dist-test-slave-h3ft krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055781, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
I20260806 22:36:21.127823 376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "bab47b7da45540eeb21f6e2d2988d633"
format_stamp: "Formatted at 2026-08-06 22:36:21 on dist-test-slave-h3ft"
server_key: "d94161f12c059984106c88d4786d4679"
server_key_iv: "97779e62e988ee865c7cbb64746f42b7"
server_key_version: "kuduclusterkey@0"
I20260806 22:36:21.129558 376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "bab47b7da45540eeb21f6e2d2988d633"
format_stamp: "Formatted at 2026-08-06 22:36:21 on dist-test-slave-h3ft"
server_key: "d94161f12c059984106c88d4786d4679"
server_key_iv: "97779e62e988ee865c7cbb64746f42b7"
server_key_version: "kuduclusterkey@0"
I20260806 22:36:21.141165 376 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.009s sys 0.001s
I20260806 22:36:21.164865 411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:36:21.167228 376 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.002s
I20260806 22:36:21.167491 376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "bab47b7da45540eeb21f6e2d2988d633"
format_stamp: "Formatted at 2026-08-06 22:36:21 on dist-test-slave-h3ft"
server_key: "d94161f12c059984106c88d4786d4679"
server_key_iv: "97779e62e988ee865c7cbb64746f42b7"
server_key_version: "kuduclusterkey@0"
I20260806 22:36:21.167826 376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:36:21.195917 376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:36:21.203173 376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:36:21.203855 376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:36:21.205618 376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:36:21.208288 376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:36:21.208423 376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:36:21.208591 376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:36:21.208662 376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:36:21.308184 376 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:46229
I20260806 22:36:21.308249 570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:46229 every 8 connection(s)
I20260806 22:36:21.309725 369 heartbeater.cc:503] Master 127.15.198.190:40181 was elected leader, sending a full tablet report...
I20260806 22:36:21.311340 376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:36:21.313647 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 376
I20260806 22:36:21.313853 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:36:21.314472 16154 external_mini_cluster.cc:1428] Setting key f36b4bdb062fb3ae3a46a2fe52476c53
Aug 06 22:36:21 dist-test-slave-h3ft krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055781, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:36:21.375391 574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:21.315884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:34727 (local address 127.15.198.190:40181)
0806 22:36:21.336956 (+ 21072us) server_negotiation.cc:206] Beginning negotiation
0806 22:36:21.336967 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:36:21.343683 (+ 6716us) server_negotiation.cc:407] Connection header received
0806 22:36:21.352345 (+ 8662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:36:21.352353 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:36:21.352501 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:36:21.352679 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:36:21.355489 (+ 2810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:21.356579 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:21.359256 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:21.359806 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:21.359981 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:21.365457 (+ 5476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:36:21.365464 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:36:21.365468 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:36:21.365514 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:36:21.370104 (+ 4590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:36:21.371546 (+ 1442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:36:21.371553 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:36:21.371557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:36:21.371762 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:36:21.372448 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:36:21.372453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:36:21.372457 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:36:21.372928 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:36:21.373081 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 22:36:21.374513 (+ 1432us) server_negotiation.cc:299] Negotiation successful
0806 22:36:21.374867 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20815,"thread_start_us":21085,"threads_started":1}
I20260806 22:36:21.379143 578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:21.317060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:40181 (local address 127.15.198.130:34727)
0806 22:36:21.343380 (+ 26320us) negotiation.cc:107] Waiting for socket to connect
0806 22:36:21.343478 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0806 22:36:21.346867 (+ 3389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:36:21.353070 (+ 6203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:36:21.353088 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:36:21.354006 (+ 918us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:36:21.355230 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:36:21.355265 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:36:21.356767 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:36:21.356774 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:36:21.359094 (+ 2320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:36:21.359111 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:36:21.360115 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:36:21.360122 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:36:21.360471 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:21.361481 (+ 1010us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:36:21.361516 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:36:21.365090 (+ 3574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:36:21.370391 (+ 5301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:36:21.370404 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:36:21.370431 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:36:21.371358 (+ 927us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:36:21.371944 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:36:21.371949 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:36:21.371953 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:36:21.372283 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:36:21.373189 (+ 906us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:36:21.373199 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:36:21.373824 (+ 625us) client_negotiation.cc:769] Sending connection context
0806 22:36:21.376294 (+ 2470us) client_negotiation.cc:240] Negotiation successful
0806 22:36:21.377166 (+ 872us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25686,"thread_start_us":29761,"threads_started":1}
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
I20260806 22:36:21.382012 572 heartbeater.cc:347] Connected to a master server at 127.15.198.190:40181
I20260806 22:36:21.382465 572 heartbeater.cc:464] Registering TS with master...
I20260806 22:36:21.383713 572 heartbeater.cc:511] Master 127.15.198.190:40181 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
I20260806 22:36:21.386193 32512 ts_manager.cc:194] Registered new tserver with Master: bab47b7da45540eeb21f6e2d2988d633 (127.15.198.130:46229)
I20260806 22:36:21.387705 32512 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:34727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
W20260806 22:36:21.440418 16154 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.198.131: real 0.119s user 0.002s sys 0.033s
I20260806 22:36:21.441589 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.198.148:33733/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:40181
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:34371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:36:22.004384 583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:36:22.004771 583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:36:22.004973 583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:36:22.013589 583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:36:22.013738 583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:36:22.013955 583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:36:22.025949 583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:34371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.198.148:33733/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:40181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:36:22.027910 583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:36:22.029668 583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:36:22.047176 592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:36:22.047561 590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:36:22.048133 589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:36:22.048647 583 server_base.cc:1034] running on GCE node
I20260806 22:36:22.049472 583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:36:22.050824 583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:36:22.052107 583 hybrid_clock.cc:648] HybridClock initialized: now 1786055782052032 us; error 60 us; skew 500 ppm
I20260806 22:36:22.052549 583 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:36:22 dist-test-slave-h3ft krb5kdc[31804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055782, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:36:22.056738 583 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:36:22.058758 583 webserver.cc:460] Webserver started at http://127.15.198.131:38309/ using document root <none> and password file <none>
I20260806 22:36:22.059752 583 fs_manager.cc:359] Metadata directory not provided
I20260806 22:36:22.059942 583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:36:22.060360 583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:36:22 dist-test-slave-h3ft krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055782, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
I20260806 22:36:22.080416 583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "ea93776913f74b4082f3ef5c4fd3c7b2"
format_stamp: "Formatted at 2026-08-06 22:36:22 on dist-test-slave-h3ft"
server_key: "f5292f1d1ab11df6f872783e436880b3"
server_key_iv: "fcb92ed3cecad5868d818313f6afac20"
server_key_version: "kuduclusterkey@0"
I20260806 22:36:22.081597 583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "ea93776913f74b4082f3ef5c4fd3c7b2"
format_stamp: "Formatted at 2026-08-06 22:36:22 on dist-test-slave-h3ft"
server_key: "f5292f1d1ab11df6f872783e436880b3"
server_key_iv: "fcb92ed3cecad5868d818313f6afac20"
server_key_version: "kuduclusterkey@0"
I20260806 22:36:22.089574 583 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260806 22:36:22.113205 604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:36:22.115556 583 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.002s
I20260806 22:36:22.115835 583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "ea93776913f74b4082f3ef5c4fd3c7b2"
format_stamp: "Formatted at 2026-08-06 22:36:22 on dist-test-slave-h3ft"
server_key: "f5292f1d1ab11df6f872783e436880b3"
server_key_iv: "fcb92ed3cecad5868d818313f6afac20"
server_key_version: "kuduclusterkey@0"
I20260806 22:36:22.116165 583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:36:22.136914 583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:36:22.142305 583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:36:22.142897 583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:36:22.144577 583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:36:22.146592 583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:36:22.146713 583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:36:22.146888 583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:36:22.146976 583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:36:22.193689 583 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:44247
I20260806 22:36:22.193724 776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:44247 every 8 connection(s)
I20260806 22:36:22.195472 583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:36:22.205037 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 583
I20260806 22:36:22.205273 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:36:22.205802 16154 external_mini_cluster.cc:1428] Setting key df030537309b37dcd25852146942aa99
Aug 06 22:36:22 dist-test-slave-h3ft krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055782, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:36:22.222214 782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:22.199211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:39105 (local address 127.15.198.190:40181)
0806 22:36:22.199932 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0806 22:36:22.199942 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:36:22.201163 (+ 1221us) server_negotiation.cc:407] Connection header received
0806 22:36:22.203395 (+ 2232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:36:22.203401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:36:22.203514 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:36:22.203654 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:36:22.205614 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.206587 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.209027 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.209818 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.209974 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:22.214374 (+ 4400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:36:22.214380 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:36:22.214383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:36:22.214429 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:36:22.217733 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:36:22.218670 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:36:22.218675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:36:22.218680 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:36:22.218825 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:36:22.219437 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:36:22.219442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:36:22.219452 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:36:22.219894 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:36:22.220068 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 22:36:22.221403 (+ 1335us) server_negotiation.cc:299] Negotiation successful
0806 22:36:22.221683 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":376,"threads_started":1}
I20260806 22:36:22.223637 784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:22.199964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:40181 (local address 127.15.198.131:39105)
0806 22:36:22.200931 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0806 22:36:22.200993 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 22:36:22.203058 (+ 2065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:36:22.203955 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:36:22.203965 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:36:22.204553 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:36:22.205403 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:36:22.205427 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.206762 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.206768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:36:22.208830 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:36:22.208856 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.210089 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.210099 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:36:22.210497 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:22.211362 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:36:22.211388 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:36:22.214082 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:36:22.217916 (+ 3834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:36:22.217928 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:36:22.217961 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:36:22.218522 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:36:22.218951 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:36:22.218956 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:36:22.218960 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:36:22.219302 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:36:22.220110 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:36:22.220119 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:36:22.220754 (+ 635us) client_negotiation.cc:769] Sending connection context
0806 22:36:22.221163 (+ 409us) client_negotiation.cc:240] Negotiation successful
0806 22:36:22.221960 (+ 797us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":463,"threads_started":1}
I20260806 22:36:22.225322 779 heartbeater.cc:347] Connected to a master server at 127.15.198.190:40181
I20260806 22:36:22.225728 779 heartbeater.cc:464] Registering TS with master...
I20260806 22:36:22.226783 779 heartbeater.cc:511] Master 127.15.198.190:40181 requested a full tablet report, sending...
I20260806 22:36:22.228718 32512 ts_manager.cc:194] Registered new tserver with Master: ea93776913f74b4082f3ef5c4fd3c7b2 (127.15.198.131:44247)
I20260806 22:36:22.229662 32512 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:39105
I20260806 22:36:22.235589 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:36:22.258405 782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:22.241365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49952 (local address 127.15.198.190:40181)
0806 22:36:22.241635 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 22:36:22.241644 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:36:22.242261 (+ 617us) server_negotiation.cc:407] Connection header received
0806 22:36:22.242511 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:36:22.242516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:36:22.242638 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:36:22.242751 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:36:22.244718 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.245612 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.247272 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.247878 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.248006 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:22.251839 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:36:22.251847 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:36:22.251853 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:36:22.251903 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:36:22.255176 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:36:22.256053 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:36:22.256059 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:36:22.256061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:36:22.256309 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:36:22.256761 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:36:22.256765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:36:22.256768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:36:22.257161 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:36:22.257392 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 22:36:22.257763 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 22:36:22.257978 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 22:36:22.263804 32512 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:36:22.273372 32512 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:36:22.308619 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:22.290905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:41995 (local address 127.0.0.1:33294)
0806 22:36:22.292001 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0806 22:36:22.292064 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 22:36:22.292246 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:36:22.294151 (+ 1905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:36:22.294163 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:36:22.294263 (+ 100us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:36:22.295426 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:36:22.295448 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.299270 (+ 3822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.299281 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:36:22.303570 (+ 4289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:36:22.303609 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.307038 (+ 3429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.307049 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:36:22.307350 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:22.307458 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 22:36:22.307632 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 22:36:22.307973 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":557,"threads_started":1}
I20260806 22:36:22.311268 803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:22.291305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33294 (local address 127.15.198.129:41995)
0806 22:36:22.293239 (+ 1934us) server_negotiation.cc:206] Beginning negotiation
0806 22:36:22.293251 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:36:22.293288 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:36:22.293421 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:36:22.293432 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:36:22.293852 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:36:22.295815 (+ 1963us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:36:22.296350 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.298934 (+ 2584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.304033 (+ 5099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.306587 (+ 2554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.310047 (+ 3460us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:22.310318 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 22:36:22.310497 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 22:36:22.310744 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1712,"thread_start_us":1464,"threads_started":1}
I20260806 22:36:22.316088 805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:22.291787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:44247 (local address 127.0.0.1:41414)
0806 22:36:22.293298 (+ 1511us) negotiation.cc:107] Waiting for socket to connect
0806 22:36:22.293317 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:36:22.293453 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:36:22.306385 (+ 12932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:36:22.306390 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:36:22.306415 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:36:22.306984 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:36:22.307027 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.310109 (+ 3082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.310115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:36:22.312763 (+ 2648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:36:22.312785 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.315074 (+ 2289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.315083 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:36:22.315293 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:22.315338 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:36:22.315423 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:36:22.315597 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1380,"thread_start_us":13877,"threads_started":1}
I20260806 22:36:22.317739 806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:22.293163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41414 (local address 127.15.198.131:44247)
0806 22:36:22.303933 (+ 10770us) server_negotiation.cc:206] Beginning negotiation
0806 22:36:22.303950 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 22:36:22.304037 (+ 87us) server_negotiation.cc:407] Connection header received
0806 22:36:22.304222 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:36:22.304237 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:36:22.304721 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:36:22.307570 (+ 2849us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:36:22.308061 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.309906 (+ 1845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.313026 (+ 3120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.314780 (+ 1754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.316765 (+ 1985us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:22.316981 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 22:36:22.317064 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 22:36:22.317260 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10411,"thread_start_us":11881,"threads_started":1}
I20260806 22:36:22.317394 32735 tablet_service.cc:1467] Processing CreateTablet for tablet 646e302527a747afabff0e7ef5eb8b92 (DEFAULT_TABLE table=test-table [id=439d53416bd44fcb8d7cce20fdb7dde4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:36:22.319625 32735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 646e302527a747afabff0e7ef5eb8b92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:36:22.323725 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:22.292556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:46229 (local address 127.0.0.1:39338)
0806 22:36:22.306017 (+ 13461us) negotiation.cc:107] Waiting for socket to connect
0806 22:36:22.306054 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:36:22.306200 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:36:22.312932 (+ 6732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:36:22.312938 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:36:22.312974 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:36:22.313385 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:36:22.313399 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.316129 (+ 2730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.316134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:36:22.318009 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:36:22.318027 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.322123 (+ 4096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.322131 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:36:22.322326 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:22.322368 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 22:36:22.323054 (+ 686us) client_negotiation.cc:240] Negotiation successful
0806 22:36:22.323237 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13278,"thread_start_us":1215,"threads_started":1}
I20260806 22:36:22.323356 672 tablet_service.cc:1467] Processing CreateTablet for tablet 646e302527a747afabff0e7ef5eb8b92 (DEFAULT_TABLE table=test-table [id=439d53416bd44fcb8d7cce20fdb7dde4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:36:22.324350 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:22.294008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39338 (local address 127.15.198.130:46229)
0806 22:36:22.309482 (+ 15474us) server_negotiation.cc:206] Beginning negotiation
0806 22:36:22.309494 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:36:22.309532 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:36:22.309674 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:36:22.309682 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:36:22.312248 (+ 2566us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:36:22.312595 (+ 347us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:36:22.313628 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.315914 (+ 2286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.318169 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.320003 (+ 1834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.322630 (+ 2627us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:22.322899 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 22:36:22.323616 (+ 717us) server_negotiation.cc:299] Negotiation successful
0806 22:36:22.323896 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15234,"thread_start_us":17719,"threads_started":1}
I20260806 22:36:22.325649 672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 646e302527a747afabff0e7ef5eb8b92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:36:22.329161 474 tablet_service.cc:1467] Processing CreateTablet for tablet 646e302527a747afabff0e7ef5eb8b92 (DEFAULT_TABLE table=test-table [id=439d53416bd44fcb8d7cce20fdb7dde4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:36:22.331344 474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 646e302527a747afabff0e7ef5eb8b92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:36:22.340106 810 tablet_bootstrap.cc:492] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1: Bootstrap starting.
I20260806 22:36:22.342638 811 tablet_bootstrap.cc:492] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2: Bootstrap starting.
I20260806 22:36:22.346724 810 tablet_bootstrap.cc:654] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1: Neither blocks nor log segments found. Creating new log.
I20260806 22:36:22.347442 811 tablet_bootstrap.cc:654] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2: Neither blocks nor log segments found. Creating new log.
I20260806 22:36:22.348330 810 log.cc:824] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:36:22.348953 811 log.cc:824] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:36:22.350750 816 tablet_bootstrap.cc:492] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633: Bootstrap starting.
I20260806 22:36:22.352560 810 tablet_bootstrap.cc:492] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1: No bootstrap required, opened a new log
I20260806 22:36:22.353024 810 ts_tablet_manager.cc:1397] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.005s
I20260806 22:36:22.353294 811 tablet_bootstrap.cc:492] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2: No bootstrap required, opened a new log
I20260806 22:36:22.353629 811 ts_tablet_manager.cc:1397] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260806 22:36:22.357486 816 tablet_bootstrap.cc:654] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633: Neither blocks nor log segments found. Creating new log.
I20260806 22:36:22.359163 816 log.cc:824] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633: Log is configured to *not* fsync() on all Append() calls
I20260806 22:36:22.362747 816 tablet_bootstrap.cc:492] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633: No bootstrap required, opened a new log
I20260806 22:36:22.363196 816 ts_tablet_manager.cc:1397] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260806 22:36:22.366348 811 raft_consensus.cc:348] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cdf1c42e5924d01a10a839644cf3ba1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 41995 } } peers { permanent_uuid: "bab47b7da45540eeb21f6e2d2988d633" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 46229 } } peers { permanent_uuid: "ea93776913f74b4082f3ef5c4fd3c7b2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44247 } }
I20260806 22:36:22.366968 811 raft_consensus.cc:374] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:36:22.367167 811 raft_consensus.cc:729] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea93776913f74b4082f3ef5c4fd3c7b2, State: Initialized, Role: FOLLOWER
I20260806 22:36:22.366971 810 raft_consensus.cc:348] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cdf1c42e5924d01a10a839644cf3ba1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 41995 } } peers { permanent_uuid: "bab47b7da45540eeb21f6e2d2988d633" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 46229 } } peers { permanent_uuid: "ea93776913f74b4082f3ef5c4fd3c7b2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44247 } }
I20260806 22:36:22.367519 810 raft_consensus.cc:374] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:36:22.367655 810 raft_consensus.cc:729] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cdf1c42e5924d01a10a839644cf3ba1, State: Initialized, Role: FOLLOWER
I20260806 22:36:22.368058 811 consensus_queue.cc:260] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cdf1c42e5924d01a10a839644cf3ba1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 41995 } } peers { permanent_uuid: "bab47b7da45540eeb21f6e2d2988d633" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 46229 } } peers { permanent_uuid: "ea93776913f74b4082f3ef5c4fd3c7b2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44247 } }
I20260806 22:36:22.368327 810 consensus_queue.cc:260] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cdf1c42e5924d01a10a839644cf3ba1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 41995 } } peers { permanent_uuid: "bab47b7da45540eeb21f6e2d2988d633" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 46229 } } peers { permanent_uuid: "ea93776913f74b4082f3ef5c4fd3c7b2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44247 } }
I20260806 22:36:22.369714 779 heartbeater.cc:503] Master 127.15.198.190:40181 was elected leader, sending a full tablet report...
I20260806 22:36:22.371207 810 ts_tablet_manager.cc:1428] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1: Time spent starting tablet: real 0.018s user 0.015s sys 0.003s
I20260806 22:36:22.372714 811 ts_tablet_manager.cc:1428] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2: Time spent starting tablet: real 0.019s user 0.016s sys 0.000s
I20260806 22:36:22.373950 816 raft_consensus.cc:348] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cdf1c42e5924d01a10a839644cf3ba1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 41995 } } peers { permanent_uuid: "bab47b7da45540eeb21f6e2d2988d633" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 46229 } } peers { permanent_uuid: "ea93776913f74b4082f3ef5c4fd3c7b2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44247 } }
I20260806 22:36:22.374509 816 raft_consensus.cc:374] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:36:22.374665 816 raft_consensus.cc:729] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab47b7da45540eeb21f6e2d2988d633, State: Initialized, Role: FOLLOWER
I20260806 22:36:22.375680 816 consensus_queue.cc:260] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cdf1c42e5924d01a10a839644cf3ba1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 41995 } } peers { permanent_uuid: "bab47b7da45540eeb21f6e2d2988d633" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 46229 } } peers { permanent_uuid: "ea93776913f74b4082f3ef5c4fd3c7b2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44247 } }
I20260806 22:36:22.378453 572 heartbeater.cc:503] Master 127.15.198.190:40181 was elected leader, sending a full tablet report...
I20260806 22:36:22.379187 816 ts_tablet_manager.cc:1428] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
W20260806 22:36:22.448009 781 tablet.cc:2369] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:36:22.519186 371 tablet.cc:2369] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:36:22.570053 819 raft_consensus.cc:482] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:36:22.570519 819 raft_consensus.cc:504] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cdf1c42e5924d01a10a839644cf3ba1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 41995 } } peers { permanent_uuid: "bab47b7da45540eeb21f6e2d2988d633" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 46229 } } peers { permanent_uuid: "ea93776913f74b4082f3ef5c4fd3c7b2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44247 } }
I20260806 22:36:22.572992 819 leader_election.cc:288] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cdf1c42e5924d01a10a839644cf3ba1 (127.15.198.129:41995), bab47b7da45540eeb21f6e2d2988d633 (127.15.198.130:46229)
I20260806 22:36:22.580408 784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:22.573519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:41995 (local address 127.15.198.131:54421)
0806 22:36:22.573775 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 22:36:22.573802 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:36:22.573953 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:36:22.574458 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:36:22.574463 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:36:22.574512 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:36:22.574768 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:36:22.574781 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.576410 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.576415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:36:22.578400 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:36:22.578417 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.579720 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.579725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:36:22.579869 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:22.579896 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 22:36:22.579986 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 22:36:22.580101 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":1}
I20260806 22:36:22.580502 803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:22.573693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:54421 (local address 127.15.198.129:41995)
0806 22:36:22.573988 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 22:36:22.574007 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 22:36:22.574054 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:36:22.574177 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:36:22.574184 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:36:22.574302 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:36:22.574432 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:36:22.574947 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.576180 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.578585 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.579549 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.579724 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:22.579818 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:36:22.580052 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 22:36:22.580210 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 22:36:22.582119 822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:22.574172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:46229 (local address 127.15.198.131:37173)
0806 22:36:22.575302 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0806 22:36:22.575318 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:36:22.575469 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:36:22.575909 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:36:22.575912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:36:22.575936 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:36:22.576407 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:36:22.576418 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.577653 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.577661 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:36:22.579555 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:36:22.579570 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.581423 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.581428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:36:22.581557 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:22.581583 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 22:36:22.581661 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:36:22.581767 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1020,"thread_start_us":833,"threads_started":1}
I20260806 22:36:22.582151 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:22.574378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:37173 (local address 127.15.198.130:46229)
0806 22:36:22.574646 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 22:36:22.574652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:36:22.575430 (+ 778us) server_negotiation.cc:407] Connection header received
0806 22:36:22.575621 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:36:22.575627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:36:22.575723 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:36:22.575839 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:36:22.576595 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.577471 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.579766 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.581186 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.581378 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:22.581471 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 22:36:22.581721 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 22:36:22.581865 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 22:36:22.582098 301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "646e302527a747afabff0e7ef5eb8b92" candidate_uuid: "ea93776913f74b4082f3ef5c4fd3c7b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cdf1c42e5924d01a10a839644cf3ba1" is_pre_election: true
I20260806 22:36:22.582976 301 raft_consensus.cc:2454] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea93776913f74b4082f3ef5c4fd3c7b2 in term 0.
I20260806 22:36:22.583534 503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "646e302527a747afabff0e7ef5eb8b92" candidate_uuid: "ea93776913f74b4082f3ef5c4fd3c7b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab47b7da45540eeb21f6e2d2988d633" is_pre_election: true
I20260806 22:36:22.583967 606 leader_election.cc:302] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cdf1c42e5924d01a10a839644cf3ba1, ea93776913f74b4082f3ef5c4fd3c7b2; no voters:
I20260806 22:36:22.584312 503 raft_consensus.cc:2454] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea93776913f74b4082f3ef5c4fd3c7b2 in term 0.
I20260806 22:36:22.584537 819 raft_consensus.cc:2781] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:36:22.584739 819 raft_consensus.cc:482] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:36:22.584892 819 raft_consensus.cc:3037] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:36:22.587957 819 raft_consensus.cc:504] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cdf1c42e5924d01a10a839644cf3ba1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 41995 } } peers { permanent_uuid: "bab47b7da45540eeb21f6e2d2988d633" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 46229 } } peers { permanent_uuid: "ea93776913f74b4082f3ef5c4fd3c7b2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44247 } }
I20260806 22:36:22.588796 819 leader_election.cc:288] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [CANDIDATE]: Term 1 election: Requested vote from peers 4cdf1c42e5924d01a10a839644cf3ba1 (127.15.198.129:41995), bab47b7da45540eeb21f6e2d2988d633 (127.15.198.130:46229)
I20260806 22:36:22.589541 301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "646e302527a747afabff0e7ef5eb8b92" candidate_uuid: "ea93776913f74b4082f3ef5c4fd3c7b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cdf1c42e5924d01a10a839644cf3ba1"
I20260806 22:36:22.589574 503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "646e302527a747afabff0e7ef5eb8b92" candidate_uuid: "ea93776913f74b4082f3ef5c4fd3c7b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab47b7da45540eeb21f6e2d2988d633"
I20260806 22:36:22.589838 301 raft_consensus.cc:3037] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:36:22.589860 503 raft_consensus.cc:3037] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:36:22.592883 301 raft_consensus.cc:2454] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea93776913f74b4082f3ef5c4fd3c7b2 in term 1.
I20260806 22:36:22.592904 503 raft_consensus.cc:2454] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea93776913f74b4082f3ef5c4fd3c7b2 in term 1.
I20260806 22:36:22.593530 606 leader_election.cc:302] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bab47b7da45540eeb21f6e2d2988d633, ea93776913f74b4082f3ef5c4fd3c7b2; no voters:
I20260806 22:36:22.593932 819 raft_consensus.cc:2781] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:36:22.594532 819 raft_consensus.cc:686] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [term 1 LEADER]: Becoming Leader. State: Replica: ea93776913f74b4082f3ef5c4fd3c7b2, State: Running, Role: LEADER
I20260806 22:36:22.595271 819 consensus_queue.cc:237] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cdf1c42e5924d01a10a839644cf3ba1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 41995 } } peers { permanent_uuid: "bab47b7da45540eeb21f6e2d2988d633" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 46229 } } peers { permanent_uuid: "ea93776913f74b4082f3ef5c4fd3c7b2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44247 } }
I20260806 22:36:22.601411 32512 catalog_manager.cc:5495] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea93776913f74b4082f3ef5c4fd3c7b2 (127.15.198.131). New cstate: current_term: 1 leader_uuid: "ea93776913f74b4082f3ef5c4fd3c7b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cdf1c42e5924d01a10a839644cf3ba1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 41995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab47b7da45540eeb21f6e2d2988d633" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 46229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea93776913f74b4082f3ef5c4fd3c7b2" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44247 } health_report { overall_health: HEALTHY } } }
W20260806 22:36:22.603413 573 tablet.cc:2369] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:36:22.622754 806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:22.615958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41426 (local address 127.15.198.131:44247)
0806 22:36:22.616155 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 22:36:22.616160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:36:22.616181 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:36:22.616337 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:36:22.616341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:36:22.616467 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:36:22.616621 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:36:22.617090 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.617917 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.619675 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.620184 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.620306 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:22.621223 (+ 917us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:36:22.622007 (+ 784us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:36:22.622201 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 22:36:22.622326 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 22:36:22.622440 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 22:36:22.633572 503 raft_consensus.cc:1261] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633 [term 1 FOLLOWER]: Refusing update from remote peer ea93776913f74b4082f3ef5c4fd3c7b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:36:22.633563 301 raft_consensus.cc:1261] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1 [term 1 FOLLOWER]: Refusing update from remote peer ea93776913f74b4082f3ef5c4fd3c7b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:36:22.634831 824 consensus_queue.cc:1035] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab47b7da45540eeb21f6e2d2988d633" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 46229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:36:22.635430 819 consensus_queue.cc:1035] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cdf1c42e5924d01a10a839644cf3ba1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 41995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:36:22.651393 828 mvcc.cc:205] Tried to move back new op lower bound from 7315684485653639168 to 7315684485515530240. Current Snapshot: MvccSnapshot[applied={T|T < 7315684485653639168}]
I20260806 22:36:22.656155 829 mvcc.cc:205] Tried to move back new op lower bound from 7315684485653639168 to 7315684485515530240. Current Snapshot: MvccSnapshot[applied={T|T < 7315684485653639168}]
I20260806 22:36:22.670902 32511 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:36:22.671432 32511 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:36:22.676852 32511 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a7713413d76402ba4f8ea2a016683b7: Sending DeleteTablet for 3 replicas of tablet 646e302527a747afabff0e7ef5eb8b92
I20260806 22:36:22.678555 672 tablet_service.cc:1514] Processing DeleteTablet for tablet 646e302527a747afabff0e7ef5eb8b92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:36:22 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:41414
I20260806 22:36:22.678603 32735 tablet_service.cc:1514] Processing DeleteTablet for tablet 646e302527a747afabff0e7ef5eb8b92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:36:22 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:33294
I20260806 22:36:22.680184 474 tablet_service.cc:1514] Processing DeleteTablet for tablet 646e302527a747afabff0e7ef5eb8b92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:36:22 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:39338
I20260806 22:36:22.680682 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 32632
I20260806 22:36:22.688469 845 tablet_replica.cc:330] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633: stopping tablet replica
I20260806 22:36:22.689314 845 raft_consensus.cc:2229] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:36:22.690192 845 raft_consensus.cc:2258] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:36:22.696528 840 tablet_replica.cc:330] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1: stopping tablet replica
I20260806 22:36:22.697430 840 raft_consensus.cc:2229] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:36:22.697775 845 ts_tablet_manager.cc:1905] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:36:22.698318 840 raft_consensus.cc:2258] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:36:22.699491 841 tablet_replica.cc:330] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2: stopping tablet replica
I20260806 22:36:22.702358 840 ts_tablet_manager.cc:1905] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:36:22.708117 845 ts_tablet_manager.cc:1918] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:36:22.708498 845 log.cc:1196] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/646e302527a747afabff0e7ef5eb8b92
I20260806 22:36:22.708609 841 raft_consensus.cc:2229] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:36:22.709345 845 ts_tablet_manager.cc:1939] T 646e302527a747afabff0e7ef5eb8b92 P bab47b7da45540eeb21f6e2d2988d633: Deleting consensus metadata
I20260806 22:36:22.709764 841 raft_consensus.cc:2258] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 06 22:36:22 dist-test-slave-h3ft krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:36:22.712214 32489 catalog_manager.cc:4841] TS bab47b7da45540eeb21f6e2d2988d633 (127.15.198.130:46229): tablet 646e302527a747afabff0e7ef5eb8b92 (table test-table [id=439d53416bd44fcb8d7cce20fdb7dde4]) successfully deleted
I20260806 22:36:22.713063 841 ts_tablet_manager.cc:1905] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:36:22.716372 840 ts_tablet_manager.cc:1918] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:36:22.716647 840 log.cc:1196] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/646e302527a747afabff0e7ef5eb8b92
I20260806 22:36:22.717410 840 ts_tablet_manager.cc:1939] T 646e302527a747afabff0e7ef5eb8b92 P 4cdf1c42e5924d01a10a839644cf3ba1: Deleting consensus metadata
I20260806 22:36:22.720402 32489 catalog_manager.cc:4841] TS 4cdf1c42e5924d01a10a839644cf3ba1 (127.15.198.129:41995): tablet 646e302527a747afabff0e7ef5eb8b92 (table test-table [id=439d53416bd44fcb8d7cce20fdb7dde4]) successfully deleted
I20260806 22:36:22.722906 841 ts_tablet_manager.cc:1918] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:36:22.723202 841 log.cc:1196] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/646e302527a747afabff0e7ef5eb8b92
I20260806 22:36:22.724021 841 ts_tablet_manager.cc:1939] T 646e302527a747afabff0e7ef5eb8b92 P ea93776913f74b4082f3ef5c4fd3c7b2: Deleting consensus metadata
I20260806 22:36:22.724792 803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:22.682107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33308 (local address 127.15.198.129:41995)
0806 22:36:22.682377 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 22:36:22.682402 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 22:36:22.693758 (+ 11356us) server_negotiation.cc:407] Connection header received
0806 22:36:22.694616 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:36:22.694628 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:36:22.694776 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:36:22.694947 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:36:22.701251 (+ 6304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.702498 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.704772 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.705533 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.705646 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:22.713487 (+ 7841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:36:22.713518 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:36:22.713543 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:36:22.713628 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:36:22.718945 (+ 5317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:36:22.720968 (+ 2023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:36:22.720983 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:36:22.720989 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:36:22.721206 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:36:22.721829 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:36:22.721836 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:36:22.721840 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:36:22.722641 (+ 801us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:36:22.723377 (+ 736us) server_negotiation.cc:1035] Waiting for connection context
0806 22:36:22.723627 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 22:36:22.724208 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 22:36:22.725775 357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:36:22.726742 32492 catalog_manager.cc:4841] TS ea93776913f74b4082f3ef5c4fd3c7b2 (127.15.198.131:44247): tablet 646e302527a747afabff0e7ef5eb8b92 (table test-table [id=439d53416bd44fcb8d7cce20fdb7dde4]) successfully deleted
I20260806 22:36:22.831162 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 32632
I20260806 22:36:22.876417 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 376
Aug 06 22:36:22 dist-test-slave-h3ft krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.130@KRBTEST.COM
I20260806 22:36:22.904440 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:22.877579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39350 (local address 127.15.198.130:46229)
0806 22:36:22.877796 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 22:36:22.877801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:36:22.877825 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:36:22.877989 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:36:22.877995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:36:22.878082 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:36:22.878177 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:36:22.880240 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.881033 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.883422 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:22.884115 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:22.884236 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:22.896131 (+ 11895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:36:22.896157 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:36:22.896182 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:36:22.896250 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:36:22.900278 (+ 4028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:36:22.901508 (+ 1230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:36:22.901521 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:36:22.901523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:36:22.901725 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:36:22.902284 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:36:22.902288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:36:22.902291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:36:22.902864 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:36:22.903276 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0806 22:36:22.903713 (+ 437us) server_negotiation.cc:299] Negotiation successful
0806 22:36:22.903991 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 22:36:22.905383 561 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:36:23.006856 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 376
I20260806 22:36:23.049503 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 583
Aug 06 22:36:23 dist-test-slave-h3ft krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.131@KRBTEST.COM
I20260806 22:36:23.074208 806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:36:23.050681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41440 (local address 127.15.198.131:44247)
0806 22:36:23.050913 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 22:36:23.050921 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:36:23.051084 (+ 163us) server_negotiation.cc:407] Connection header received
0806 22:36:23.051357 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:36:23.051363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:36:23.051490 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:36:23.051618 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:36:23.053839 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:23.055214 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:23.057353 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:36:23.058087 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:36:23.058226 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:36:23.064843 (+ 6617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:36:23.064873 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:36:23.064903 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:36:23.064979 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:36:23.069850 (+ 4871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:36:23.070869 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:36:23.070889 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:36:23.070893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:36:23.071369 (+ 476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:36:23.071923 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:36:23.071930 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:36:23.071933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:36:23.072641 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:36:23.073000 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0806 22:36:23.073416 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 22:36:23.073735 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 22:36:23.075356 768 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:36:23.221626 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 583
I20260806 22:36:23.253353 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 32463
I20260806 22:36:23.255060 32553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:36:23.389552 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 32463
I20260806 22:36:23.427754 16154 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 22:36:23.777398 16154 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 22:36:23.777787 16154 mini_ranger.cc:66] Stopping Ranger...
I20260806 22:36:24.136412 16154 mini_ranger.cc:68] Stopped Ranger
2026-08-06 22:36:24.136 UTC [31841] LOG: received smart shutdown request
2026-08-06 22:36:24.141 UTC [31841] LOG: background worker "logical replication launcher" (PID 31855) exited with exit code 1
2026-08-06 22:36:24.142 UTC [31845] LOG: shutting down
2026-08-06 22:36:24.336 UTC [31841] LOG: database system is shut down
2026-08-06T22:36:24Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (61951 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:36:24 dist-test-slave-h3ft krb5kdc[861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:36:24 dist-test-slave-h3ft krb5kdc[861](info): set up 2 sockets
Aug 06 22:36:24 dist-test-slave-h3ft krb5kdc[861](info): commencing operation
krb5kdc: starting...
W20260806 22:36:25.895925 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.358s user 0.003s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:36:26 dist-test-slave-h3ft krb5kdc[861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:36:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:36:26Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.198.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.198.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.198.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/rangeradmin_127.15.198.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.198.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/rangeradmin_127.15.198.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.198.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.198.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.198.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/rangerlookup_127.15.198.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.198.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/rangerlookup_127.15.198.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.198.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.198.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.198.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/HTTP_127.15.198.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.198.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/HTTP_127.15.198.148@KRBTEST.COM.keytab.
I20260806 22:36:26.435025 16154 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskuRfOoI/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/postgres -l logfile start
2026-08-06 22:36:29.513 UTC [901] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 22:36:29.513 UTC [901] LOG: listening on IPv4 address "127.15.198.148", port 37757
2026-08-06 22:36:29.518 UTC [901] LOG: listening on Unix socket "/tmp/.s.PGSQL.37757"
2026-08-06 22:36:29.548 UTC [909] LOG: database system was shut down at 2026-08-06 22:36:27 UTC
2026-08-06 22:36:29.553 UTC [901] LOG: database system is ready to accept connections
I20260806 22:36:30.786245 16154 mini_postgres.cc:96] Postgres bound to 37757
2026-08-06 22:36:30.817 UTC [922] FATAL: database "slave" does not exist
127.15.198.148:37757 - accepting connections
I20260806 22:36:30.818583 16154 mini_ranger.cc:161] Starting Ranger...
I20260806 22:36:30.865411 16154 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 22:36:31.512936 16154 mini_ranger.cc:87] Created ranger Postgres database
I20260806 22:36:31.513125 16154 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-admin
2026-08-06 22:36:31,872 [I] DB FLAVOR :POSTGRES
2026-08-06 22:36:31,872 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:36:31,872 [I] Checking connection..
2026-08-06 22:36:31,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 22:36:32,177 [I] Checking connection passed.
2026-08-06 22:36:32,178 [I] --------- Verifying version history table ---------
2026-08-06 22:36:32,178 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:36:32,478 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:36:32,478 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:36:32,795 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:36:32,795 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 22:36:32,795 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 22:36:33.087 UTC [1018] WARNING: there is no transaction in progress
2026-08-06 22:36:33,099 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 22:36:33,100 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:36:33,404 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 22:36:33,404 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:36:33,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 22:36:33,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 22:36:33,999 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h3ft.c.gcp-upstream.internal','N') ;"
2026-08-06 22:36:34,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:36:34,580 [I] Table x_portal_user does not exist in database ranger
2026-08-06 22:36:34,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:36:34,893 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 22:36:34,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:36:35,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:36:35,491 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 22:36:35,491 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 22:36:36.709 UTC [1190] WARNING: there is no transaction in progress
2026-08-06 22:36:36.730 UTC [1190] WARNING: there is no transaction in progress
2026-08-06 22:36:36.753 UTC [1190] WARNING: there is no transaction in progress
2026-08-06 22:36:36.773 UTC [1190] WARNING: there is no transaction in progress
2026-08-06 22:36:36.795 UTC [1190] WARNING: there is no transaction in progress
2026-08-06 22:36:36.819 UTC [1190] WARNING: there is no transaction in progress
2026-08-06 22:36:36.893 UTC [1190] WARNING: there is no transaction in progress
2026-08-06 22:36:36.901 UTC [1190] WARNING: there is no transaction in progress
2026-08-06 22:36:36.918 UTC [1190] WARNING: there is no transaction in progress
2026-08-06 22:36:36.930 UTC [1190] WARNING: there is no transaction in progress
2026-08-06 22:36:36.943 UTC [1190] WARNING: there is no transaction in progress
2026-08-06 22:36:36.960 UTC [1190] WARNING: there is no transaction in progress
2026-08-06 22:36:36.972 UTC [1190] WARNING: there is no transaction in progress
2026-08-06 22:36:36.979 UTC [1190] WARNING: there is no transaction in progress
2026-08-06 22:36:36.988 UTC [1190] WARNING: there is no transaction in progress
2026-08-06 22:36:36.996 UTC [1190] WARNING: there is no transaction in progress
2026-08-06 22:36:37,822 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 22:36:37,822 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 22:36:38,100 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 22:36:38,100 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:36:38,408 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 22:36:38,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:36:38,711 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 22:36:38,711 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:36:39,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:36:39,320 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h3ft.c.gcp-upstream.internal';"
2026-08-06 22:36:39,616 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 22:36:39,616 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 22:36:39,915 [I] DB_PATCHES have already been applied
I20260806 22:36:39.923804 16154 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-admin:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskuRfOoI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 22:36:39.923972 16154 mini_ranger.cc:193] Using host: 127.15.198.148
I20260806 22:36:39.952704 16154 mini_ranger.cc:227] Ranger admin URL: http://127.15.198.148:42599
Aug 06, 2026 10:36:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:36:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:36:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 10:36:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 10:36:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 10:36:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 10:36:41 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 22:36:41.523469 16154 mini_ranger.cc:160] Time spent starting Ranger: real 10.705s user 0.008s sys 0.080s
Aug 06, 2026 10:37:05 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 10:37:05 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 10:37:05 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 10:37:06 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 10:37:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 10:37:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 10:37:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 10:37:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 10:37:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 10:37:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 10:37:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 10:37:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 10:37:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 10:37:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 10:37:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 10:37:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 10:37:06 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 10:37:07 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 22:37:08.038486 16154 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.198.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.198.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.198.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/rangerkms_127.15.198.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.198.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/rangerkms_127.15.198.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.198.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.198.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.198.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/HTTP_127.15.198.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.198.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/HTTP_127.15.198.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 22:37:08 dist-test-slave-h3ft krb5kdc[861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055828, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 22:37:08.319774 16154 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 22:37:08.358183 16154 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 22:37:09.292371 16154 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 22:37:09.292528 16154 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-kms
I20260806 22:37:09.292595 16154 mini_ranger_kms.cc:227] Using postgres at 127.15.198.148:37757
2026-08-06 22:37:09,765 [I] DB FLAVOR :POSTGRES
2026-08-06 22:37:09,766 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:37:09,766 [I] Checking connection
2026-08-06 22:37:09,766 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 22:37:10,084 [I] connection success
2026-08-06 22:37:10,085 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 22:37:10,085 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 22:37:10,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 22:37:10,365 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 22:37:10,366 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:37:10,366 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 22:37:10,366 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.198.148:37757/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 22:37:10,674 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 22:37:11.287294 16154 mini_ranger_kms.cc:310] Created kms service
I20260806 22:37:11.479157 16154 mini_ranger_kms.cc:326] Created kudu user
I20260806 22:37:11.537617 16154 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 22:37:11 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055828, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
I20260806 22:37:12.080206 16154 mini_ranger_kms.cc:384] Added ranger policy
I20260806 22:37:12.080724 16154 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-kms:/tmp/dist-test-taskuRfOoI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskuRfOoI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 22:37:12.080808 16154 mini_ranger_kms.cc:240] Using host: 127.15.198.148
I20260806 22:37:12.111306 16154 mini_ranger_kms.cc:276] Ranger KMS PID: 1461
I20260806 22:37:12.111511 16154 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.198.148:54343
Aug 06, 2026 10:37:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:37:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:37:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 10:37:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 10:37:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 10:37:13 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 22:37:14 dist-test-slave-h3ft krb5kdc[861](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786055834, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.198.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:15.102445 16154 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.783s user 0.077s sys 0.450s
I20260806 22:37:15.102771 16154 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 22:37:15.102974 16154 mini_ranger_kms.cc:401] 127.15.198.148:54343/kms/v1/keys
Aug 06 22:37:15 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055834, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.198.148@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
Aug 06 22:37:15 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055834, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.198.148@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
Aug 06 22:37:18 dist-test-slave-h3ft krb5kdc[861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:37:18.512450 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:41177
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:41177
--ranger_config_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.198.148:54343/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:18.970614 1531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:18.971189 1531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:18.971382 1531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:18.979940 1531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:18.980150 1531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:18.980276 1531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:18.980392 1531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:18.980491 1531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:18.993255 1531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.198.148:54343/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:41177
--ranger_config_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:41177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:18.995793 1531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:18.997618 1531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:19.013011 1539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:19.013069 1537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:19.012988 1536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:19.014104 1531 server_base.cc:1034] running on GCE node
I20260806 22:37:19.015365 1531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:19.017433 1531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:19.018704 1531 hybrid_clock.cc:648] HybridClock initialized: now 1786055839018663 us; error 45 us; skew 500 ppm
I20260806 22:37:19.019348 1531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:19 dist-test-slave-h3ft krb5kdc[861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055839, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:19.023589 1531 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:37:19.025792 1531 webserver.cc:460] Webserver started at http://127.15.198.190:43529/ using document root <none> and password file <none>
I20260806 22:37:19.026732 1531 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:19.026911 1531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:19.027343 1531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:37:19 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055839, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
I20260806 22:37:19.110544 1531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "c1f2d398c6ab4d5fbf95a12d35531e35"
format_stamp: "Formatted at 2026-08-06 22:37:19 on dist-test-slave-h3ft"
server_key: "43c895b565a22d406f538b9d8c0dc640"
server_key_iv: "ab4039e5b7313d1123151a633e136a09"
server_key_version: "kuduclusterkey@0"
I20260806 22:37:19.111809 1531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "c1f2d398c6ab4d5fbf95a12d35531e35"
format_stamp: "Formatted at 2026-08-06 22:37:19 on dist-test-slave-h3ft"
server_key: "43c895b565a22d406f538b9d8c0dc640"
server_key_iv: "ab4039e5b7313d1123151a633e136a09"
server_key_version: "kuduclusterkey@0"
I20260806 22:37:19.119613 1531 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 22:37:19.148129 1547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:19.150477 1531 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:37:19.150741 1531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "c1f2d398c6ab4d5fbf95a12d35531e35"
format_stamp: "Formatted at 2026-08-06 22:37:19 on dist-test-slave-h3ft"
server_key: "43c895b565a22d406f538b9d8c0dc640"
server_key_iv: "ab4039e5b7313d1123151a633e136a09"
server_key_version: "kuduclusterkey@0"
I20260806 22:37:19.151130 1531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:19.202731 1531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:19.208531 1531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:19.208946 1531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:19.232277 1531 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:41177
I20260806 22:37:19.232311 1621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:41177 every 8 connection(s)
I20260806 22:37:19.234120 1531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:19.239387 1623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:19.242422 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 1531
I20260806 22:37:19.242715 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:37:19.243577 16154 external_mini_cluster.cc:1428] Setting key 69e2bf9f4f88076a4579a1b7a627ec6a
I20260806 22:37:19.254750 1623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35: Bootstrap starting.
I20260806 22:37:19.260107 1623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:19.261757 1623 log.cc:824] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35: Log is configured to *not* fsync() on all Append() calls
Aug 06 22:37:19 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:37:19.265621 1623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35: No bootstrap required, opened a new log
I20260806 22:37:19.272253 1628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:19.246550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50854 (local address 127.15.198.190:41177)
0806 22:37:19.247639 (+ 1089us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:19.247654 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:19.247712 (+ 58us) server_negotiation.cc:407] Connection header received
0806 22:37:19.249849 (+ 2137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:19.249888 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:19.250455 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:19.250917 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:19.252989 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:19.254564 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:19.256586 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:19.257450 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:19.257742 (+ 292us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:19.263382 (+ 5640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:19.263392 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:19.263412 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:19.263465 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:19.267106 (+ 3641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:19.267945 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:19.267953 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:19.267960 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:19.268140 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:19.268605 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:19.268609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:19.268611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:19.269313 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:19.269702 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:19.270212 (+ 510us) server_negotiation.cc:299] Negotiation successful
0806 22:37:19.270764 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":551,"threads_started":1}
I20260806 22:37:19.275099 1623 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f2d398c6ab4d5fbf95a12d35531e35" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41177 } }
I20260806 22:37:19.275693 1623 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:19.275818 1623 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1f2d398c6ab4d5fbf95a12d35531e35, State: Initialized, Role: FOLLOWER
I20260806 22:37:19.276584 1623 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f2d398c6ab4d5fbf95a12d35531e35" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41177 } }
I20260806 22:37:19.276834 1623 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:19.276937 1623 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:19.277088 1623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:19.279767 1623 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f2d398c6ab4d5fbf95a12d35531e35" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41177 } }
I20260806 22:37:19.280529 1623 leader_election.cc:302] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1f2d398c6ab4d5fbf95a12d35531e35; no voters:
I20260806 22:37:19.281191 1623 leader_election.cc:288] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:19.281378 1631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:19.282270 1631 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [term 1 LEADER]: Becoming Leader. State: Replica: c1f2d398c6ab4d5fbf95a12d35531e35, State: Running, Role: LEADER
I20260806 22:37:19.283041 1631 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f2d398c6ab4d5fbf95a12d35531e35" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41177 } }
I20260806 22:37:19.283963 1623 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:19.287196 1635 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1f2d398c6ab4d5fbf95a12d35531e35. Latest consensus state: current_term: 1 leader_uuid: "c1f2d398c6ab4d5fbf95a12d35531e35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f2d398c6ab4d5fbf95a12d35531e35" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41177 } } }
I20260806 22:37:19.287683 1635 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:19.288623 1633 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1f2d398c6ab4d5fbf95a12d35531e35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f2d398c6ab4d5fbf95a12d35531e35" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41177 } } }
I20260806 22:37:19.289243 1633 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:19.289834 1642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:19.297959 1642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:19.298972 1623 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 22:37:19.324723 1642 catalog_manager.cc:1347] Generated new cluster ID: ee6b87dc66e94feb93f56059b4d35ba3
I20260806 22:37:19.324955 1642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:19.349821 1642 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:19.351851 1642 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:19.363188 1642 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35: Generated new TSK 0
I20260806 22:37:19.364730 1642 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
I20260806 22:37:20.281147 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.198.148:54343/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:41177
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:37:20 dist-test-slave-h3ft krb5kdc[861](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786055840, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 22:37:20.853755 1693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:20.854557 1693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:20.854856 1693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:20.869513 1693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:20.869926 1693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:20.870321 1693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:37:20.887789 1693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.198.148:54343/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:41177
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:20.889907 1693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:20.892382 1693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:20.913826 1705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:20.914130 1703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:20.914479 1710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:20.915578 1693 server_base.cc:1034] running on GCE node
I20260806 22:37:20.916584 1693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:20.918176 1693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:20.919454 1693 hybrid_clock.cc:648] HybridClock initialized: now 1786055840919369 us; error 64 us; skew 500 ppm
I20260806 22:37:20.919916 1693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:20 dist-test-slave-h3ft krb5kdc[861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055840, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:20.925058 1693 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:37:20.927525 1693 webserver.cc:460] Webserver started at http://127.15.198.129:33965/ using document root <none> and password file <none>
I20260806 22:37:20.928746 1693 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:20.929003 1693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:20.929490 1693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:37:20 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055840, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
I20260806 22:37:20.955832 1693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "69625ba1ec6f4914880887a38221051b"
format_stamp: "Formatted at 2026-08-06 22:37:20 on dist-test-slave-h3ft"
server_key: "e13f7e29f74a64e84674e86beb48b647"
server_key_iv: "f466432f0397af6fde8451dab3e1749e"
server_key_version: "kuduclusterkey@0"
I20260806 22:37:20.957274 1693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "69625ba1ec6f4914880887a38221051b"
format_stamp: "Formatted at 2026-08-06 22:37:20 on dist-test-slave-h3ft"
server_key: "e13f7e29f74a64e84674e86beb48b647"
server_key_iv: "f466432f0397af6fde8451dab3e1749e"
server_key_version: "kuduclusterkey@0"
I20260806 22:37:20.967890 1693 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.005s sys 0.004s
I20260806 22:37:21.007980 1721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:21.010975 1693 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.009s sys 0.004s
I20260806 22:37:21.011404 1693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "69625ba1ec6f4914880887a38221051b"
format_stamp: "Formatted at 2026-08-06 22:37:20 on dist-test-slave-h3ft"
server_key: "e13f7e29f74a64e84674e86beb48b647"
server_key_iv: "f466432f0397af6fde8451dab3e1749e"
server_key_version: "kuduclusterkey@0"
I20260806 22:37:21.011900 1693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:21.040612 1693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:21.046919 1693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:21.047492 1693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:21.049118 1693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:21.051321 1693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:21.051446 1693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:21.051626 1693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:21.051715 1693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:21.141819 1693 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:43549
I20260806 22:37:21.141831 1876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:43549 every 8 connection(s)
I20260806 22:37:21.144620 1693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:37:21.150339 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 1693
I20260806 22:37:21.150616 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:37:21.151616 16154 external_mini_cluster.cc:1428] Setting key cb155403dd604ec26c5ec241c1629c6d
Aug 06 22:37:21 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055840, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:37:21.193979 1880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:21.148596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:38633 (local address 127.15.198.190:41177)
0806 22:37:21.167579 (+ 18983us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:21.167587 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:21.167618 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:37:21.167714 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:21.167721 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:21.167853 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:21.168018 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:21.171084 (+ 3066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:21.172484 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:21.175520 (+ 3036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:21.176217 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:21.176335 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:21.183976 (+ 7641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:21.183985 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:21.183990 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:21.184048 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:21.188038 (+ 3990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:21.190151 (+ 2113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:21.190160 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:21.190164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:21.190380 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:21.191128 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:21.191133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:21.191137 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:21.191577 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:21.191788 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:21.193194 (+ 1406us) server_negotiation.cc:299] Negotiation successful
0806 22:37:21.193508 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18776,"thread_start_us":21144,"threads_started":1}
I20260806 22:37:21.196960 1881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:21.149593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41177 (local address 127.15.198.129:38633)
0806 22:37:21.160618 (+ 11025us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:21.160708 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:21.164218 (+ 3510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:21.168660 (+ 4442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:21.168677 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:21.169588 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:21.170754 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:21.170792 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:21.172766 (+ 1974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:21.172771 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:21.175254 (+ 2483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:21.175273 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:21.176808 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:21.176817 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:21.177199 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:21.178331 (+ 1132us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:21.178373 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:21.182113 (+ 3740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:21.188296 (+ 6183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:21.188310 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:21.188339 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:21.189149 (+ 810us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:21.190563 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:21.190568 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:21.190571 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:21.190893 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:21.191889 (+ 996us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:21.191900 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:21.192556 (+ 656us) client_negotiation.cc:769] Sending connection context
0806 22:37:21.194400 (+ 1844us) client_negotiation.cc:240] Negotiation successful
0806 22:37:21.195168 (+ 768us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10593,"thread_start_us":9056,"threads_started":1}
I20260806 22:37:21.199592 1877 heartbeater.cc:347] Connected to a master server at 127.15.198.190:41177
I20260806 22:37:21.200035 1877 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:21.201229 1877 heartbeater.cc:511] Master 127.15.198.190:41177 requested a full tablet report, sending...
I20260806 22:37:21.204818 1570 ts_manager.cc:194] Registered new tserver with Master: 69625ba1ec6f4914880887a38221051b (127.15.198.129:43549)
I20260806 22:37:21.207607 1570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:38633
Aug 06 22:37:21 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055840, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
W20260806 22:37:21.282661 16154 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.198.130: real 0.110s user 0.000s sys 0.032s
I20260806 22:37:21.283962 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.198.148:54343/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:41177
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:37:21 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055840, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
W20260806 22:37:21.961203 1886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:21.961949 1886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:21.962240 1886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:21.975312 1886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:21.975641 1886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:21.975947 1886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:37:21.993829 1886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.198.148:54343/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:41177
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:21.996598 1886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:21.999085 1886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:22.045295 1897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:22.047482 1899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:22.057130 1886 server_base.cc:1034] running on GCE node
W20260806 22:37:22.059506 1903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:22.067525 1886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:22.079530 1886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:22.089768 1886 hybrid_clock.cc:648] HybridClock initialized: now 1786055842089587 us; error 159 us; skew 500 ppm
I20260806 22:37:22.090502 1886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:22 dist-test-slave-h3ft krb5kdc[861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055842, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:22.101053 1886 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:37:22.155037 1886 webserver.cc:460] Webserver started at http://127.15.198.130:46523/ using document root <none> and password file <none>
I20260806 22:37:22.156783 1886 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:22.157132 1886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:22.157735 1886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:37:22 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055842, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
I20260806 22:37:22.188067 1886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "e364792037ca4b7aa9a667c89c9c7455"
format_stamp: "Formatted at 2026-08-06 22:37:22 on dist-test-slave-h3ft"
server_key: "546150ec87ad9f1a18dbdf71240f2a4e"
server_key_iv: "63560ea12b1546d83d24beacb3ad1c07"
server_key_version: "kuduclusterkey@0"
I20260806 22:37:22.200266 1886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "e364792037ca4b7aa9a667c89c9c7455"
format_stamp: "Formatted at 2026-08-06 22:37:22 on dist-test-slave-h3ft"
server_key: "546150ec87ad9f1a18dbdf71240f2a4e"
server_key_iv: "63560ea12b1546d83d24beacb3ad1c07"
server_key_version: "kuduclusterkey@0"
I20260806 22:37:22.212908 1877 heartbeater.cc:503] Master 127.15.198.190:41177 was elected leader, sending a full tablet report...
I20260806 22:37:22.233655 1886 fs_manager.cc:691] Time spent creating directory manager: real 0.032s user 0.001s sys 0.011s
I20260806 22:37:22.273770 1913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:22.276683 1886 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.002s sys 0.012s
I20260806 22:37:22.277088 1886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "e364792037ca4b7aa9a667c89c9c7455"
format_stamp: "Formatted at 2026-08-06 22:37:22 on dist-test-slave-h3ft"
server_key: "546150ec87ad9f1a18dbdf71240f2a4e"
server_key_iv: "63560ea12b1546d83d24beacb3ad1c07"
server_key_version: "kuduclusterkey@0"
I20260806 22:37:22.277544 1886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:22.331563 1886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:22.341572 1886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:22.342110 1886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:22.355793 1886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:22.368345 1886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:22.368585 1886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:22.368775 1886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:22.368875 1886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:22.709825 1886 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:36771
I20260806 22:37:22.709844 2099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:36771 every 8 connection(s)
I20260806 22:37:22.712100 1886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:37:22.713732 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 1886
I20260806 22:37:22.713943 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:37:22.714557 16154 external_mini_cluster.cc:1428] Setting key 7e4b7ac6ad87b53032f1f55b0e250064
Aug 06 22:37:22 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055842, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
I20260806 22:37:22.763217 2104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:22.722861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:34885 (local address 127.15.198.190:41177)
0806 22:37:22.738659 (+ 15798us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:22.738667 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:22.738694 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:37:22.738777 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:22.738782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:22.738905 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:22.739084 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:22.741754 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.743211 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.746235 (+ 3024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.747033 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.747206 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:22.753226 (+ 6020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:22.753236 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:22.753244 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:22.753305 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:22.758152 (+ 4847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:22.759599 (+ 1447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:22.759608 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:22.759611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:22.759820 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:22.760518 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:22.760526 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:22.760532 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:22.761000 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:22.761168 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:22.762432 (+ 1264us) server_negotiation.cc:299] Negotiation successful
0806 22:37:22.762721 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15633,"thread_start_us":15441,"threads_started":1}
I20260806 22:37:22.764626 2107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:22.723740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41177 (local address 127.15.198.130:34885)
0806 22:37:22.733497 (+ 9757us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:22.733587 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:22.736506 (+ 2919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:22.739489 (+ 2983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:22.739509 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:22.740334 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:22.741478 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:22.741510 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.743437 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.743457 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:22.745991 (+ 2534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:22.746027 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.747624 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.747634 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:22.748028 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:22.749059 (+ 1031us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:22.749106 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:22.752826 (+ 3720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:22.758423 (+ 5597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:22.758438 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:22.758470 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:22.759396 (+ 926us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:22.759994 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:22.760000 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:22.760007 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:22.760350 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:22.761277 (+ 927us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:22.761289 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:22.761918 (+ 629us) client_negotiation.cc:769] Sending connection context
0806 22:37:22.762290 (+ 372us) client_negotiation.cc:240] Negotiation successful
0806 22:37:22.762992 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9275,"thread_start_us":13191,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
I20260806 22:37:22.766959 2100 heartbeater.cc:347] Connected to a master server at 127.15.198.190:41177
I20260806 22:37:22.767413 2100 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:22.768637 2100 heartbeater.cc:511] Master 127.15.198.190:41177 requested a full tablet report, sending...
I20260806 22:37:22.771028 1570 ts_manager.cc:194] Registered new tserver with Master: e364792037ca4b7aa9a667c89c9c7455 (127.15.198.130:36771)
I20260806 22:37:22.772506 1570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:34885
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
I20260806 22:37:22.812019 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.198.148:54343/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:41177
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:23.261761 2113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:23.262161 2113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:23.262329 2113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:23.270730 2113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:23.270892 2113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:23.271190 2113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:37:23.282939 2113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.198.148:54343/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:41177
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:23.285115 2113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:23.286780 2113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:23.303632 2122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:23.303597 2125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:23.303681 2123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:23.304848 2113 server_base.cc:1034] running on GCE node
I20260806 22:37:23.305626 2113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:23.306833 2113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:23.308121 2113 hybrid_clock.cc:648] HybridClock initialized: now 1786055843308056 us; error 71 us; skew 500 ppm
I20260806 22:37:23.308558 2113 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:23 dist-test-slave-h3ft krb5kdc[861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055843, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:23.312810 2113 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:37:23.314772 2113 webserver.cc:460] Webserver started at http://127.15.198.131:38321/ using document root <none> and password file <none>
I20260806 22:37:23.315747 2113 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:23.315908 2113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:23.316316 2113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:37:23 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055843, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
I20260806 22:37:23.338150 2113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "e920a28a4bfc4a3591c63ae13603766e"
format_stamp: "Formatted at 2026-08-06 22:37:23 on dist-test-slave-h3ft"
server_key: "0e5e209efb4b64bfb4eb1dd867886a57"
server_key_iv: "19a26fbc850c80097b4eb8d01a0ee410"
server_key_version: "kuduclusterkey@0"
I20260806 22:37:23.339623 2113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "e920a28a4bfc4a3591c63ae13603766e"
format_stamp: "Formatted at 2026-08-06 22:37:23 on dist-test-slave-h3ft"
server_key: "0e5e209efb4b64bfb4eb1dd867886a57"
server_key_iv: "19a26fbc850c80097b4eb8d01a0ee410"
server_key_version: "kuduclusterkey@0"
I20260806 22:37:23.348465 2113 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.005s
I20260806 22:37:23.370617 2135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:23.372614 2113 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 22:37:23.372869 2113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "e920a28a4bfc4a3591c63ae13603766e"
format_stamp: "Formatted at 2026-08-06 22:37:23 on dist-test-slave-h3ft"
server_key: "0e5e209efb4b64bfb4eb1dd867886a57"
server_key_iv: "19a26fbc850c80097b4eb8d01a0ee410"
server_key_version: "kuduclusterkey@0"
I20260806 22:37:23.373211 2113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:23.396579 2113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:23.401827 2113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:23.402222 2113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:23.403553 2113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:23.405357 2113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:23.405486 2113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:23.405679 2113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:23.405757 2113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:23.446677 2113 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:36469
I20260806 22:37:23.446736 2307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:36469 every 8 connection(s)
I20260806 22:37:23.448473 2113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:37:23.455809 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 2113
I20260806 22:37:23.456054 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:37:23.456772 16154 external_mini_cluster.cc:1428] Setting key 24740ab4d1614e959ec137f24da2407d
Aug 06 22:37:23 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055843, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:37:23.477134 2311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.452066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:49303 (local address 127.15.198.190:41177)
0806 22:37:23.452758 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.452765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.454203 (+ 1438us) server_negotiation.cc:407] Connection header received
0806 22:37:23.457210 (+ 3007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.457216 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.457340 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.457459 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:23.459771 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.461018 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.463100 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.463624 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.463754 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.468447 (+ 4693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:23.468456 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:23.468459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:23.468520 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:23.472299 (+ 3779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.473920 (+ 1621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.473926 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.473930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.474216 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.474927 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.474931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.474933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.475396 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:23.475542 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.476551 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.476811 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":317,"threads_started":1}
I20260806 22:37:23.478835 2314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.452898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41177 (local address 127.15.198.131:49303)
0806 22:37:23.453957 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:23.454011 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:23.456711 (+ 2700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:23.457893 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:23.457910 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:23.458504 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:23.459579 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.459604 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.461239 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.461244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.462763 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.462791 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.463950 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.463957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.464266 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.465236 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:23.465269 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:23.468020 (+ 2751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:23.472665 (+ 4645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:23.472685 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:23.472717 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:23.473736 (+ 1019us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:23.474384 (+ 648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:23.474388 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:23.474390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:23.474790 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:23.475544 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:23.475552 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:23.476079 (+ 527us) client_negotiation.cc:769] Sending connection context
0806 22:37:23.476412 (+ 333us) client_negotiation.cc:240] Negotiation successful
0806 22:37:23.476974 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":403,"threads_started":1}
I20260806 22:37:23.480800 2308 heartbeater.cc:347] Connected to a master server at 127.15.198.190:41177
I20260806 22:37:23.481305 2308 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:23.482375 2308 heartbeater.cc:511] Master 127.15.198.190:41177 requested a full tablet report, sending...
I20260806 22:37:23.484660 1570 ts_manager.cc:194] Registered new tserver with Master: e920a28a4bfc4a3591c63ae13603766e (127.15.198.131:36469)
I20260806 22:37:23.486029 1570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:49303
I20260806 22:37:23.488029 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:37:23.490684 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 1693
Aug 06 22:37:23 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:37:23.512603 2317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.492127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33468 (local address 127.15.198.129:43549)
0806 22:37:23.493031 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.493040 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.493076 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:37:23.493158 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.493166 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.493438 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.493613 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:23.495538 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.496851 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.498481 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.499315 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.499465 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.505098 (+ 5633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:23.505121 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:23.505145 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:23.505201 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:23.509034 (+ 3833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.510086 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.510096 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.510099 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.510292 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.510711 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.510715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.510717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.511232 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:23.511564 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.511956 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.512232 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":479,"threads_started":1}
I20260806 22:37:23.514598 1872 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:23.610240 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 1693
I20260806 22:37:23.642426 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 1886
Aug 06 22:37:23 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.130@KRBTEST.COM
I20260806 22:37:23.663548 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.643715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39090 (local address 127.15.198.130:36771)
0806 22:37:23.644364 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.644373 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.644415 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:37:23.644500 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.644508 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.644817 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.644990 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:23.646525 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.647864 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.649356 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.650037 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.650147 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.655894 (+ 5747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:23.655914 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:23.655936 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:23.655989 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:23.659846 (+ 3857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.660849 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.660858 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.660861 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.661079 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.661542 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.661546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.661548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.662005 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:23.662276 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.662681 (+ 405us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.663082 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":327,"threads_started":1}
I20260806 22:37:23.665381 2093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:23.756690 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 1886
I20260806 22:37:23.786352 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 2113
Aug 06 22:37:23 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.131@KRBTEST.COM
I20260806 22:37:23.808424 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.787676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56196 (local address 127.15.198.131:36469)
0806 22:37:23.788569 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.788577 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.788617 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:37:23.788702 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.788711 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.789077 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.789252 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:23.790943 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.792384 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.794292 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.795129 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.795257 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.800910 (+ 5653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:23.800930 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:23.800953 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:23.801008 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:23.804739 (+ 3731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.805780 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.805790 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.805794 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.805961 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.806430 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.806434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.806436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.806879 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:23.807320 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.807715 (+ 395us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.807983 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":475,"threads_started":1}
I20260806 22:37:23.810252 2303 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:23.905332 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 2113
I20260806 22:37:23.934553 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 1531
I20260806 22:37:23.936443 1616 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:24.062642 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 1531
I20260806 22:37:24.102221 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:41177
--webserver_interface=127.15.198.190
--webserver_port=43529
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:41177
--ranger_config_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.198.148:54343/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.198.148:54343/kms with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:24.531332 2328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:24.531733 2328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:24.531865 2328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:24.540272 2328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:24.540441 2328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:24.540534 2328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:24.540642 2328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:24.540735 2328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:24.552826 2328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.198.148:54343/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:41177
--ranger_config_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:41177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=43529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:24.554765 2328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:24.556651 2328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:24.571928 2335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:24.572078 2336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:24.572141 2341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:24.573351 2328 server_base.cc:1034] running on GCE node
I20260806 22:37:24.574648 2328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:24.576910 2328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:24.578197 2328 hybrid_clock.cc:648] HybridClock initialized: now 1786055844578138 us; error 48 us; skew 500 ppm
I20260806 22:37:24.578776 2328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:24 dist-test-slave-h3ft krb5kdc[861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055844, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:24.583307 2328 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:37:24.585449 2328 webserver.cc:460] Webserver started at http://127.15.198.190:43529/ using document root <none> and password file <none>
I20260806 22:37:24.586318 2328 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:24.586465 2328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:37:24 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055844, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
I20260806 22:37:24.679991 2328 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 22:37:24.684383 2354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:24.686411 2328 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:37:24.686658 2328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "c1f2d398c6ab4d5fbf95a12d35531e35"
format_stamp: "Formatted at 2026-08-06 22:37:19 on dist-test-slave-h3ft"
server_key: "43c895b565a22d406f538b9d8c0dc640"
server_key_iv: "ab4039e5b7313d1123151a633e136a09"
server_key_version: "kuduclusterkey@0"
I20260806 22:37:24.688086 2328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:24.722821 2328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:24.728504 2328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:24.728824 2328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:24.752419 2328 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:41177
I20260806 22:37:24.752429 2427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:41177 every 8 connection(s)
I20260806 22:37:24.754426 2328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:24.756832 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 2328
I20260806 22:37:24.757880 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.198.148:54343/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:43549
--local_ip_for_outbound_sockets=127.15.198.129
--tserver_master_addrs=127.15.198.190:41177
--webserver_port=33965
--webserver_interface=127.15.198.129
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.198.148:54343/kms with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:37:24.763990 2431 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:37:24.768290 2431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35: Bootstrap starting.
I20260806 22:37:24.793083 2431 log.cc:824] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:24.806265 2431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:37:24.807267 2431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35: Bootstrap complete.
I20260806 22:37:24.818339 2431 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f2d398c6ab4d5fbf95a12d35531e35" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41177 } }
I20260806 22:37:24.819547 2431 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1f2d398c6ab4d5fbf95a12d35531e35, State: Initialized, Role: FOLLOWER
I20260806 22:37:24.820417 2431 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f2d398c6ab4d5fbf95a12d35531e35" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41177 } }
I20260806 22:37:24.820715 2431 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:24.820873 2431 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:24.821092 2431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:37:24.826036 2431 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f2d398c6ab4d5fbf95a12d35531e35" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41177 } }
I20260806 22:37:24.826798 2431 leader_election.cc:302] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1f2d398c6ab4d5fbf95a12d35531e35; no voters:
I20260806 22:37:24.827728 2431 leader_election.cc:288] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:37:24.827919 2438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:37:24.829617 2438 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [term 2 LEADER]: Becoming Leader. State: Replica: c1f2d398c6ab4d5fbf95a12d35531e35, State: Running, Role: LEADER
I20260806 22:37:24.830472 2438 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f2d398c6ab4d5fbf95a12d35531e35" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41177 } }
I20260806 22:37:24.830905 2431 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:24.834312 2440 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c1f2d398c6ab4d5fbf95a12d35531e35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f2d398c6ab4d5fbf95a12d35531e35" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41177 } } }
I20260806 22:37:24.834946 2440 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:24.839248 2442 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1f2d398c6ab4d5fbf95a12d35531e35. Latest consensus state: current_term: 2 leader_uuid: "c1f2d398c6ab4d5fbf95a12d35531e35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f2d398c6ab4d5fbf95a12d35531e35" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41177 } } }
I20260806 22:37:24.840554 2442 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:24.843572 2448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:24.859601 2448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:24.862223 2448 catalog_manager.cc:1259] Loaded cluster ID: ee6b87dc66e94feb93f56059b4d35ba3
I20260806 22:37:24.862519 2448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:24.867736 2448 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:24.870136 2448 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35: Loaded TSK: 0
I20260806 22:37:24.872653 2448 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:37:24.879518 2431 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260806 22:37:25.324100 2433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:25.324833 2433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:25.325105 2433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:25.338696 2433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:25.339118 2433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:25.339466 2433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:37:25.356047 2433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.198.148:54343/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:43549
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=33965
--tserver_master_addrs=127.15.198.190:41177
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:25.358808 2433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:25.361162 2433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:25.393888 2493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:25.394928 2496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:25.400915 2433 server_base.cc:1034] running on GCE node
W20260806 22:37:25.401523 2499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:25.402760 2433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:25.405202 2433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:25.406610 2433 hybrid_clock.cc:648] HybridClock initialized: now 1786055845406507 us; error 85 us; skew 500 ppm
I20260806 22:37:25.407270 2433 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:25 dist-test-slave-h3ft krb5kdc[861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055845, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:25.413475 2433 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:37:25.416527 2433 webserver.cc:460] Webserver started at http://127.15.198.129:33965/ using document root <none> and password file <none>
I20260806 22:37:25.417994 2433 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:25.418246 2433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:37:25 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055845, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
I20260806 22:37:25.486080 2433 fs_manager.cc:709] Time spent opening directory manager: real 0.017s user 0.007s sys 0.005s
I20260806 22:37:25.491201 2511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:25.494094 2433 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260806 22:37:25.494521 2433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "69625ba1ec6f4914880887a38221051b"
format_stamp: "Formatted at 2026-08-06 22:37:20 on dist-test-slave-h3ft"
server_key: "e13f7e29f74a64e84674e86beb48b647"
server_key_iv: "f466432f0397af6fde8451dab3e1749e"
server_key_version: "kuduclusterkey@0"
I20260806 22:37:25.496636 2433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:25.530246 2433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:25.536322 2433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:25.536705 2433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:25.538153 2433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:25.540488 2433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:25.540637 2433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:25.540803 2433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:25.540899 2433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:25.632520 2433 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:43549
I20260806 22:37:25.632570 2679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:43549 every 8 connection(s)
I20260806 22:37:25.634613 2433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:37:25.638526 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 2433
I20260806 22:37:25.639655 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.198.148:54343/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:36771
--local_ip_for_outbound_sockets=127.15.198.130
--tserver_master_addrs=127.15.198.190:41177
--webserver_port=46523
--webserver_interface=127.15.198.130
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.198.148:54343/kms with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:37:25 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055845, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:37:25.680173 2684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.640154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:35485 (local address 127.15.198.190:41177)
0806 22:37:25.641287 (+ 1133us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:25.641316 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:25.642468 (+ 1152us) server_negotiation.cc:407] Connection header received
0806 22:37:25.646911 (+ 4443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:25.646949 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:25.647764 (+ 815us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:25.648325 (+ 561us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:25.651917 (+ 3592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.654271 (+ 2354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.657587 (+ 3316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.658902 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.659181 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.665826 (+ 6645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:25.665857 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:25.665889 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:25.665969 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:25.671473 (+ 5504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.673138 (+ 1665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.673155 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.673169 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.673482 (+ 313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.674303 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.674313 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.674319 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.675331 (+ 1012us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:25.675590 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:25.677155 (+ 1565us) server_negotiation.cc:299] Negotiation successful
0806 22:37:25.677888 (+ 733us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":1061,"threads_started":1}
I20260806 22:37:25.684880 2688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.641035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41177 (local address 127.15.198.129:35485)
0806 22:37:25.642166 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:25.642250 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:25.645695 (+ 3445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:25.649523 (+ 3828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:25.649540 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:25.650437 (+ 897us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:25.651555 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.651593 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.654625 (+ 3032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.654633 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.657293 (+ 2660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.657318 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.659833 (+ 2515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.659855 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.660240 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.661322 (+ 1082us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:25.661367 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:25.665276 (+ 3909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:25.671834 (+ 6558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:25.671867 (+ 33us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:25.671899 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:25.672852 (+ 953us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:25.673695 (+ 843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:25.673704 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:25.673710 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:25.674112 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:25.675701 (+ 1589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:25.675714 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:25.676356 (+ 642us) client_negotiation.cc:769] Sending connection context
0806 22:37:25.681729 (+ 5373us) client_negotiation.cc:240] Negotiation successful
0806 22:37:25.682720 (+ 991us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":554,"threads_started":1}
I20260806 22:37:25.690379 2682 heartbeater.cc:347] Connected to a master server at 127.15.198.190:41177
I20260806 22:37:25.690824 2682 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:25.692061 2682 heartbeater.cc:511] Master 127.15.198.190:41177 requested a full tablet report, sending...
W20260806 22:37:25.697079 2682 heartbeater.cc:644] Failed to heartbeat to 127.15.198.190:41177 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:37:25.697481 2682 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:25.698050 2682 heartbeater.cc:511] Master 127.15.198.190:41177 requested a full tablet report, sending...
I20260806 22:37:25.700122 2682 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:25.700567 2682 heartbeater.cc:511] Master 127.15.198.190:41177 requested a full tablet report, sending...
W20260806 22:37:25.702008 2682 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:37:26.181432 2691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:26.182137 2691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:26.182379 2691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:26.195436 2691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:26.195756 2691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:26.196046 2691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:37:26.211753 2691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.198.148:54343/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:36771
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=46523
--tserver_master_addrs=127.15.198.190:41177
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:26.214126 2691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:26.216344 2691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:26.250427 2714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:26.252591 2713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:26.260831 2717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:26.263887 2691 server_base.cc:1034] running on GCE node
I20260806 22:37:26.265072 2691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:26.266815 2691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:26.268283 2691 hybrid_clock.cc:648] HybridClock initialized: now 1786055846268198 us; error 89 us; skew 500 ppm
I20260806 22:37:26.268793 2691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:26 dist-test-slave-h3ft krb5kdc[861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055846, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:26.274195 2691 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:37:26.276841 2691 webserver.cc:460] Webserver started at http://127.15.198.130:46523/ using document root <none> and password file <none>
I20260806 22:37:26.278141 2691 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:26.278347 2691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:37:26 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055846, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
I20260806 22:37:26.313249 2691 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.005s sys 0.001s
I20260806 22:37:26.320861 2729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:26.323546 2691 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.004s sys 0.000s
I20260806 22:37:26.323932 2691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "e364792037ca4b7aa9a667c89c9c7455"
format_stamp: "Formatted at 2026-08-06 22:37:22 on dist-test-slave-h3ft"
server_key: "546150ec87ad9f1a18dbdf71240f2a4e"
server_key_iv: "63560ea12b1546d83d24beacb3ad1c07"
server_key_version: "kuduclusterkey@0"
I20260806 22:37:26.325852 2691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 06 22:37:26 dist-test-slave-h3ft krb5kdc[861](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786055846, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:26.357295 2691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:26.364233 2691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:26.364727 2691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:26.371146 2691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:26.373379 2691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:26.373512 2691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:26.373636 2691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:26.373709 2691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:26.429287 2691 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:36771
I20260806 22:37:26.429327 2892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:36771 every 8 connection(s)
I20260806 22:37:26.431471 2691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:37:26.432616 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 2691
I20260806 22:37:26.433586 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.198.148:54343/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:36469
--local_ip_for_outbound_sockets=127.15.198.131
--tserver_master_addrs=127.15.198.190:41177
--webserver_port=38321
--webserver_interface=127.15.198.131
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.198.148:54343/kms with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:37:26 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055846, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:37:26.489347 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.450138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41177 (local address 127.15.198.130:56643)
0806 22:37:26.458636 (+ 8498us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:26.458712 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:26.460969 (+ 2257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:26.462128 (+ 1159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:26.462147 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:26.463058 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:26.464228 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.464265 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.466198 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.466207 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.468784 (+ 2577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.468807 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.470529 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.470540 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.470943 (+ 403us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.473343 (+ 2400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:26.473404 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:26.477432 (+ 4028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:26.482696 (+ 5264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:26.482714 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:26.482746 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:26.483679 (+ 933us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:26.484246 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:26.484252 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:26.484255 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:26.484632 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:26.485538 (+ 906us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:26.485549 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:26.486193 (+ 644us) client_negotiation.cc:769] Sending connection context
0806 22:37:26.486629 (+ 436us) client_negotiation.cc:240] Negotiation successful
0806 22:37:26.487463 (+ 834us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8079,"thread_start_us":7755,"threads_started":1}
I20260806 22:37:26.490939 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.451095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:56643 (local address 127.15.198.190:41177)
0806 22:37:26.457797 (+ 6702us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:26.457808 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:26.459432 (+ 1624us) server_negotiation.cc:407] Connection header received
0806 22:37:26.461395 (+ 1963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:26.461402 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:26.461545 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:26.461701 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:26.464544 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.465912 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.469046 (+ 3134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.469892 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.470062 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.477916 (+ 7854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:26.477929 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:26.477935 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:26.478002 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:26.482410 (+ 4408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:26.483900 (+ 1490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:26.483908 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:26.483912 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:26.484092 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:26.484793 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:26.484798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:26.484802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:26.485241 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:26.485445 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:26.490110 (+ 4665us) server_negotiation.cc:299] Negotiation successful
0806 22:37:26.490480 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6458,"thread_start_us":6296,"threads_started":1}
I20260806 22:37:26.494277 2893 heartbeater.cc:347] Connected to a master server at 127.15.198.190:41177
I20260806 22:37:26.494813 2893 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:26.496345 2893 heartbeater.cc:511] Master 127.15.198.190:41177 requested a full tablet report, sending...
I20260806 22:37:26.499680 2379 ts_manager.cc:194] Registered new tserver with Master: e364792037ca4b7aa9a667c89c9c7455 (127.15.198.130:36771)
I20260806 22:37:26.502820 2379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:56643
I20260806 22:37:26.702245 2682 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:26.702957 2682 heartbeater.cc:511] Master 127.15.198.190:41177 requested a full tablet report, sending...
I20260806 22:37:26.705029 2379 ts_manager.cc:194] Registered new tserver with Master: 69625ba1ec6f4914880887a38221051b (127.15.198.129:43549)
I20260806 22:37:26.706472 2379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:35485
Aug 06 22:37:26 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055846, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
W20260806 22:37:27.002992 2895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:27.003620 2895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:27.003804 2895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:27.015260 2895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:27.015558 2895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:27.015796 2895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:37:27.032366 2895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.198.148:54343/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:36469
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=38321
--tserver_master_addrs=127.15.198.190:41177
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:27.034910 2895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:27.037142 2895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:27.067493 2905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:27.068476 2895 server_base.cc:1034] running on GCE node
W20260806 22:37:27.069255 2904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:27.069855 2910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:27.070951 2895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:27.072713 2895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:27.079149 2895 hybrid_clock.cc:648] HybridClock initialized: now 1786055847079042 us; error 73 us; skew 500 ppm
I20260806 22:37:27.079784 2895 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:27 dist-test-slave-h3ft krb5kdc[861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055847, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:27.085388 2895 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:37:27.096441 2895 webserver.cc:460] Webserver started at http://127.15.198.131:38321/ using document root <none> and password file <none>
I20260806 22:37:27.097921 2895 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:27.098212 2895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:37:27 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055847, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
I20260806 22:37:27.141304 2895 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.004s sys 0.000s
I20260806 22:37:27.148072 2921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:27.150512 2895 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.003s
I20260806 22:37:27.150766 2895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "e920a28a4bfc4a3591c63ae13603766e"
format_stamp: "Formatted at 2026-08-06 22:37:23 on dist-test-slave-h3ft"
server_key: "0e5e209efb4b64bfb4eb1dd867886a57"
server_key_iv: "19a26fbc850c80097b4eb8d01a0ee410"
server_key_version: "kuduclusterkey@0"
I20260806 22:37:27.152618 2895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:27.194444 2895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:27.199856 2895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:27.200242 2895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:27.201642 2895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:27.204013 2895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:27.204150 2895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:27.204308 2895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:27.204413 2895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:37:27 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055846, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for HTTP/127.15.198.148@KRBTEST.COM
I20260806 22:37:27.497166 2895 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:36469
I20260806 22:37:27.497330 3085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:36469 every 8 connection(s)
I20260806 22:37:27.500138 2895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:37:27.501238 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 2895
I20260806 22:37:27.509418 2893 heartbeater.cc:503] Master 127.15.198.190:41177 was elected leader, sending a full tablet report...
Aug 06 22:37:27 dist-test-slave-h3ft krb5kdc[861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055847, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:37:27.557376 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:27.502919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50858 (local address 127.15.198.190:41177)
0806 22:37:27.519804 (+ 16885us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:27.519813 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:27.519843 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:37:27.519951 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:27.519958 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:27.520085 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:27.520246 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:27.530916 (+ 10670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.532399 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.535074 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.535642 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.535824 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:27.540095 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:27.540103 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:27.540107 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:27.540182 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:27.553332 (+ 13150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:27.554621 (+ 1289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:27.554627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:27.554633 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:27.554841 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:27.555407 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:27.555415 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:27.555422 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:27.555901 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:27.556235 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:27.556592 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 22:37:27.556884 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16698,"thread_start_us":17458,"threads_started":1}
I20260806 22:37:27.561893 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:27.521100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:34051 (local address 127.15.198.190:41177)
0806 22:37:27.535392 (+ 14292us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:27.535399 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:27.535424 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:37:27.537173 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:27.537180 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:27.537311 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:27.537462 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:27.540355 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.541847 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.544730 (+ 2883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.545437 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.545597 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:27.551190 (+ 5593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:27.551204 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:27.551210 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:27.551271 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:27.555679 (+ 4408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:27.556904 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:27.556908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:27.556912 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:27.557048 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:27.557995 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:27.558001 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:27.558004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:27.558575 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:27.558729 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:27.561108 (+ 2379us) server_negotiation.cc:299] Negotiation successful
0806 22:37:27.561433 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14135,"thread_start_us":13899,"threads_started":1}
I20260806 22:37:27.563611 3094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:27.520178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41177 (local address 127.15.198.131:34051)
0806 22:37:27.531523 (+ 11345us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:27.531612 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:27.536684 (+ 5072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:27.537874 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:27.537891 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:27.538722 (+ 831us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:27.540033 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:27.540066 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.542095 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.542102 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:27.544497 (+ 2395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:27.544529 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.546037 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.546047 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:27.546395 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:27.547323 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:27.547368 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:27.550722 (+ 3354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:27.555880 (+ 5158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:27.555893 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:27.555920 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:27.556738 (+ 818us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:27.557399 (+ 661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:27.557407 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:27.557414 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:27.557843 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:27.558939 (+ 1096us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:27.558949 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:27.559917 (+ 968us) client_negotiation.cc:769] Sending connection context
0806 22:37:27.560390 (+ 473us) client_negotiation.cc:240] Negotiation successful
0806 22:37:27.561820 (+ 1430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10808,"thread_start_us":13271,"threads_started":1}
I20260806 22:37:27.565963 3089 heartbeater.cc:347] Connected to a master server at 127.15.198.190:41177
I20260806 22:37:27.566395 3089 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:27.567699 3089 heartbeater.cc:511] Master 127.15.198.190:41177 requested a full tablet report, sending...
I20260806 22:37:27.570791 2379 ts_manager.cc:194] Registered new tserver with Master: e920a28a4bfc4a3591c63ae13603766e (127.15.198.131:36469)
I20260806 22:37:27.572311 2379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:34051
I20260806 22:37:27.580536 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:37:27.627521 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:27.605795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50866 (local address 127.15.198.190:41177)
0806 22:37:27.606094 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:27.606102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:27.606908 (+ 806us) server_negotiation.cc:407] Connection header received
0806 22:37:27.607297 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:27.607305 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:27.607439 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:27.607548 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:27.610250 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.611450 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.613815 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.614563 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.614740 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:27.618895 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:27.618904 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:27.618911 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:27.618972 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:27.623248 (+ 4276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:27.624422 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:27.624430 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:27.624434 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:27.624685 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:27.625297 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:27.625303 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:27.625307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:27.625800 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:27.626106 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:27.626680 (+ 574us) server_negotiation.cc:299] Negotiation successful
0806 22:37:27.627084 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 22:37:27.637753 2379 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:37:27.650067 2379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:27.710804 2682 heartbeater.cc:503] Master 127.15.198.190:41177 was elected leader, sending a full tablet report...
I20260806 22:37:27.729620 3125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:27.675791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:36771 (local address 127.0.0.1:39098)
0806 22:37:27.715023 (+ 39232us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:27.715047 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:27.715185 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:27.718286 (+ 3101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:27.718295 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:27.718332 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:27.718875 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:27.718891 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.721168 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.721174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:27.724387 (+ 3213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:27.724411 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.728407 (+ 3996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.728414 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:27.728670 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:27.728745 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 22:37:27.728906 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 22:37:27.729179 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39002,"thread_start_us":39475,"threads_started":1}
I20260806 22:37:27.729681 3124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:27.676641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39098 (local address 127.15.198.130:36771)
0806 22:37:27.716970 (+ 40329us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:27.716982 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:27.717022 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:37:27.717163 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:27.717172 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:27.717584 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:27.717840 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:27.719227 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.720979 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.725064 (+ 4085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.726771 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.727023 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:27.727266 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:27.729037 (+ 1771us) server_negotiation.cc:299] Negotiation successful
0806 22:37:27.729273 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40092,"thread_start_us":46375,"threads_started":1}
I20260806 22:37:27.736588 3126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:27.693217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56208 (local address 127.15.198.131:36469)
0806 22:37:27.713498 (+ 20281us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:27.713513 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:27.726054 (+ 12541us) server_negotiation.cc:407] Connection header received
0806 22:37:27.726627 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:27.726639 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:27.727049 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:27.727305 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:27.728480 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.730085 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.732784 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.734481 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.734659 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:27.734894 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:27.735878 (+ 984us) server_negotiation.cc:299] Negotiation successful
0806 22:37:27.736102 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19959,"thread_start_us":20287,"threads_started":1}
I20260806 22:37:27.737493 3121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:27.692671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:36469 (local address 127.0.0.1:56208)
0806 22:37:27.725847 (+ 33176us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:27.725872 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:27.726210 (+ 338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:27.727755 (+ 1545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:27.727759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:27.727786 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:27.728212 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:27.728230 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.730266 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.730269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:27.732559 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:27.732599 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.735278 (+ 2679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.735284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:27.735490 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:27.735660 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 22:37:27.736854 (+ 1194us) client_negotiation.cc:240] Negotiation successful
0806 22:37:27.737034 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33006,"mutex_wait_us":101,"thread_start_us":23489,"threads_started":1}
I20260806 22:37:27.736567 2801 tablet_service.cc:1467] Processing CreateTablet for tablet 18bd16ac0a844df087f25bda5c8dd65c (DEFAULT_TABLE table=test-table [id=ec1adb71c8ea43a2bd308583cd3d0c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:27.739794 2801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18bd16ac0a844df087f25bda5c8dd65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:27.742786 3122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:27.675586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:43549 (local address 127.0.0.1:33474)
0806 22:37:27.713788 (+ 38202us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:27.713860 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:27.714185 (+ 325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:27.730421 (+ 16236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:27.730428 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:27.730483 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:27.730909 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:27.730921 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.734907 (+ 3986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.734915 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:27.739220 (+ 4305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:27.739235 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.740915 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.740922 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:27.741116 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:27.741154 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:37:27.741323 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 22:37:27.741509 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37776,"thread_start_us":49874,"threads_started":1}
I20260806 22:37:27.743620 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:27.676073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33474 (local address 127.15.198.129:43549)
0806 22:37:27.727604 (+ 51531us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:27.727615 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:27.727652 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:37:27.727771 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:27.727780 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:27.728048 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:27.730436 (+ 2388us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:27.732976 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.734724 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.739434 (+ 4710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.740653 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.740915 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:27.741160 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:27.741878 (+ 718us) server_negotiation.cc:299] Negotiation successful
0806 22:37:27.743293 (+ 1415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51304,"thread_start_us":48801,"threads_started":1}
I20260806 22:37:27.742849 2990 tablet_service.cc:1467] Processing CreateTablet for tablet 18bd16ac0a844df087f25bda5c8dd65c (DEFAULT_TABLE table=test-table [id=ec1adb71c8ea43a2bd308583cd3d0c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:27.744896 2990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18bd16ac0a844df087f25bda5c8dd65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:27.747951 2580 tablet_service.cc:1467] Processing CreateTablet for tablet 18bd16ac0a844df087f25bda5c8dd65c (DEFAULT_TABLE table=test-table [id=ec1adb71c8ea43a2bd308583cd3d0c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:27.750540 2580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18bd16ac0a844df087f25bda5c8dd65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:27.765918 3127 tablet_bootstrap.cc:492] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455: Bootstrap starting.
I20260806 22:37:27.768890 3128 tablet_bootstrap.cc:492] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e: Bootstrap starting.
I20260806 22:37:27.773373 3128 tablet_bootstrap.cc:654] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:27.773423 3127 tablet_bootstrap.cc:654] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:27.775077 3127 log.cc:824] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:27.775060 3128 log.cc:824] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:27.783880 3129 tablet_bootstrap.cc:492] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b: Bootstrap starting.
I20260806 22:37:27.787662 3128 tablet_bootstrap.cc:492] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e: No bootstrap required, opened a new log
I20260806 22:37:27.788311 3128 ts_tablet_manager.cc:1397] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.010s
I20260806 22:37:27.795269 3127 tablet_bootstrap.cc:492] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455: No bootstrap required, opened a new log
I20260806 22:37:27.795363 3129 tablet_bootstrap.cc:654] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:27.795888 3127 ts_tablet_manager.cc:1397] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.022s
I20260806 22:37:27.796978 3129 log.cc:824] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:27.800827 3128 raft_consensus.cc:348] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69625ba1ec6f4914880887a38221051b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 43549 } } peers { permanent_uuid: "e364792037ca4b7aa9a667c89c9c7455" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36771 } } peers { permanent_uuid: "e920a28a4bfc4a3591c63ae13603766e" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36469 } }
I20260806 22:37:27.801607 3128 raft_consensus.cc:374] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:27.801782 3128 raft_consensus.cc:729] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e920a28a4bfc4a3591c63ae13603766e, State: Initialized, Role: FOLLOWER
I20260806 22:37:27.802819 3128 consensus_queue.cc:260] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69625ba1ec6f4914880887a38221051b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 43549 } } peers { permanent_uuid: "e364792037ca4b7aa9a667c89c9c7455" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36771 } } peers { permanent_uuid: "e920a28a4bfc4a3591c63ae13603766e" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36469 } }
I20260806 22:37:27.809535 3127 raft_consensus.cc:348] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69625ba1ec6f4914880887a38221051b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 43549 } } peers { permanent_uuid: "e364792037ca4b7aa9a667c89c9c7455" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36771 } } peers { permanent_uuid: "e920a28a4bfc4a3591c63ae13603766e" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36469 } }
I20260806 22:37:27.811978 3127 raft_consensus.cc:374] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:27.812121 3127 raft_consensus.cc:729] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e364792037ca4b7aa9a667c89c9c7455, State: Initialized, Role: FOLLOWER
I20260806 22:37:27.813373 3089 heartbeater.cc:503] Master 127.15.198.190:41177 was elected leader, sending a full tablet report...
I20260806 22:37:27.813321 3127 consensus_queue.cc:260] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69625ba1ec6f4914880887a38221051b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 43549 } } peers { permanent_uuid: "e364792037ca4b7aa9a667c89c9c7455" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36771 } } peers { permanent_uuid: "e920a28a4bfc4a3591c63ae13603766e" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36469 } }
I20260806 22:37:27.815255 3128 ts_tablet_manager.cc:1428] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e: Time spent starting tablet: real 0.027s user 0.025s sys 0.000s
I20260806 22:37:27.817688 3127 ts_tablet_manager.cc:1428] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455: Time spent starting tablet: real 0.022s user 0.012s sys 0.006s
I20260806 22:37:27.818888 3129 tablet_bootstrap.cc:492] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b: No bootstrap required, opened a new log
I20260806 22:37:27.819384 3129 ts_tablet_manager.cc:1397] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b: Time spent bootstrapping tablet: real 0.036s user 0.014s sys 0.010s
I20260806 22:37:27.831404 3129 raft_consensus.cc:348] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69625ba1ec6f4914880887a38221051b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 43549 } } peers { permanent_uuid: "e364792037ca4b7aa9a667c89c9c7455" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36771 } } peers { permanent_uuid: "e920a28a4bfc4a3591c63ae13603766e" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36469 } }
I20260806 22:37:27.832219 3129 raft_consensus.cc:374] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:27.832365 3129 raft_consensus.cc:729] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69625ba1ec6f4914880887a38221051b, State: Initialized, Role: FOLLOWER
I20260806 22:37:27.833316 3129 consensus_queue.cc:260] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69625ba1ec6f4914880887a38221051b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 43549 } } peers { permanent_uuid: "e364792037ca4b7aa9a667c89c9c7455" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36771 } } peers { permanent_uuid: "e920a28a4bfc4a3591c63ae13603766e" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36469 } }
I20260806 22:37:27.835537 3138 raft_consensus.cc:482] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:27.836136 3129 ts_tablet_manager.cc:1428] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b: Time spent starting tablet: real 0.017s user 0.010s sys 0.005s
I20260806 22:37:27.836017 3138 raft_consensus.cc:504] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69625ba1ec6f4914880887a38221051b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 43549 } } peers { permanent_uuid: "e364792037ca4b7aa9a667c89c9c7455" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36771 } } peers { permanent_uuid: "e920a28a4bfc4a3591c63ae13603766e" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36469 } }
I20260806 22:37:27.839959 3138 leader_election.cc:288] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e364792037ca4b7aa9a667c89c9c7455 (127.15.198.130:36771), e920a28a4bfc4a3591c63ae13603766e (127.15.198.131:36469)
I20260806 22:37:27.850925 3124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:27.839499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:41669 (local address 127.15.198.130:36771)
0806 22:37:27.840359 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:27.840366 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:27.842996 (+ 2630us) server_negotiation.cc:407] Connection header received
0806 22:37:27.843186 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:27.843191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:27.843298 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:27.843406 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:27.844275 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.845167 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.848301 (+ 3134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.849737 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.849938 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:27.850031 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:27.850425 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 22:37:27.850590 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679}
I20260806 22:37:27.851090 3140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:27.838843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:36771 (local address 127.15.198.129:41669)
0806 22:37:27.842778 (+ 3935us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:27.842803 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:27.842973 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:27.843441 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:27.843446 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:27.843477 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:27.844035 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:27.844050 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.845321 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.845329 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:27.848088 (+ 2759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:27.848110 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.850006 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.850013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:27.850218 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:27.850259 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:37:27.850386 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 22:37:27.850551 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3770,"mutex_wait_us":33,"thread_start_us":3551,"threads_started":1}
I20260806 22:37:27.851203 3126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:27.839521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:41995 (local address 127.15.198.131:36469)
0806 22:37:27.839809 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:27.839819 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:27.843283 (+ 3464us) server_negotiation.cc:407] Connection header received
0806 22:37:27.843446 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:27.843456 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:27.843549 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:27.843663 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:27.844476 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.845567 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.848837 (+ 3270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.849904 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.850084 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:27.850171 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:27.850728 (+ 557us) server_negotiation.cc:299] Negotiation successful
0806 22:37:27.850895 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 22:37:27.852099 3139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:27.838775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:36469 (local address 127.15.198.129:41995)
0806 22:37:27.843168 (+ 4393us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:27.843185 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:27.843294 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:27.843704 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:27.843709 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:27.843737 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:27.844275 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:27.844287 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.846181 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.846187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:27.848645 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:27.848657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.850240 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.850246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:27.850416 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:27.850463 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 22:37:27.851519 (+ 1056us) client_negotiation.cc:240] Negotiation successful
0806 22:37:27.851687 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1938,"mutex_wait_us":26,"thread_start_us":1706,"threads_started":1}
I20260806 22:37:27.853248 3020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18bd16ac0a844df087f25bda5c8dd65c" candidate_uuid: "69625ba1ec6f4914880887a38221051b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e920a28a4bfc4a3591c63ae13603766e" is_pre_election: true
I20260806 22:37:27.853257 2831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18bd16ac0a844df087f25bda5c8dd65c" candidate_uuid: "69625ba1ec6f4914880887a38221051b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e364792037ca4b7aa9a667c89c9c7455" is_pre_election: true
I20260806 22:37:27.853977 3020 raft_consensus.cc:2454] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69625ba1ec6f4914880887a38221051b in term 0.
I20260806 22:37:27.853982 2831 raft_consensus.cc:2454] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69625ba1ec6f4914880887a38221051b in term 0.
I20260806 22:37:27.854982 2513 leader_election.cc:302] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 69625ba1ec6f4914880887a38221051b, e364792037ca4b7aa9a667c89c9c7455, e920a28a4bfc4a3591c63ae13603766e; no voters:
I20260806 22:37:27.855789 3138 raft_consensus.cc:2781] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:27.856007 3138 raft_consensus.cc:482] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:27.856153 3138 raft_consensus.cc:3037] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:27.859860 3138 raft_consensus.cc:504] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69625ba1ec6f4914880887a38221051b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 43549 } } peers { permanent_uuid: "e364792037ca4b7aa9a667c89c9c7455" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36771 } } peers { permanent_uuid: "e920a28a4bfc4a3591c63ae13603766e" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36469 } }
I20260806 22:37:27.860792 3138 leader_election.cc:288] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [CANDIDATE]: Term 1 election: Requested vote from peers e364792037ca4b7aa9a667c89c9c7455 (127.15.198.130:36771), e920a28a4bfc4a3591c63ae13603766e (127.15.198.131:36469)
I20260806 22:37:27.861523 2831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18bd16ac0a844df087f25bda5c8dd65c" candidate_uuid: "69625ba1ec6f4914880887a38221051b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e364792037ca4b7aa9a667c89c9c7455"
I20260806 22:37:27.861572 3020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18bd16ac0a844df087f25bda5c8dd65c" candidate_uuid: "69625ba1ec6f4914880887a38221051b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e920a28a4bfc4a3591c63ae13603766e"
I20260806 22:37:27.861845 3020 raft_consensus.cc:3037] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:27.861846 2831 raft_consensus.cc:3037] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:27.866214 2831 raft_consensus.cc:2454] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69625ba1ec6f4914880887a38221051b in term 1.
I20260806 22:37:27.866214 3020 raft_consensus.cc:2454] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69625ba1ec6f4914880887a38221051b in term 1.
I20260806 22:37:27.866997 2515 leader_election.cc:302] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69625ba1ec6f4914880887a38221051b, e920a28a4bfc4a3591c63ae13603766e; no voters:
I20260806 22:37:27.867507 3138 raft_consensus.cc:2781] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:27.868247 3138 raft_consensus.cc:686] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [term 1 LEADER]: Becoming Leader. State: Replica: 69625ba1ec6f4914880887a38221051b, State: Running, Role: LEADER
I20260806 22:37:27.868993 3138 consensus_queue.cc:237] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69625ba1ec6f4914880887a38221051b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 43549 } } peers { permanent_uuid: "e364792037ca4b7aa9a667c89c9c7455" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36771 } } peers { permanent_uuid: "e920a28a4bfc4a3591c63ae13603766e" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36469 } }
I20260806 22:37:27.875890 2378 catalog_manager.cc:5495] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b reported cstate change: term changed from 0 to 1, leader changed from <none> to 69625ba1ec6f4914880887a38221051b (127.15.198.129). New cstate: current_term: 1 leader_uuid: "69625ba1ec6f4914880887a38221051b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69625ba1ec6f4914880887a38221051b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 43549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e364792037ca4b7aa9a667c89c9c7455" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e920a28a4bfc4a3591c63ae13603766e" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36469 } health_report { overall_health: UNKNOWN } } }
W20260806 22:37:27.889197 2683 tablet.cc:2369] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:37:27.906854 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:27.899134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33476 (local address 127.15.198.129:43549)
0806 22:37:27.899383 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:27.899389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:27.899411 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:37:27.899711 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:27.899717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:27.899895 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:27.900076 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:27.900634 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.901519 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.903765 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.904244 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:27.904355 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:27.905072 (+ 717us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:27.906081 (+ 1009us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:27.906252 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:27.906388 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 22:37:27.906514 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 22:37:27.918704 3020 raft_consensus.cc:1261] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e [term 1 FOLLOWER]: Refusing update from remote peer 69625ba1ec6f4914880887a38221051b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:27.918740 2831 raft_consensus.cc:1261] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455 [term 1 FOLLOWER]: Refusing update from remote peer 69625ba1ec6f4914880887a38221051b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:27.919958 3138 consensus_queue.cc:1035] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e920a28a4bfc4a3591c63ae13603766e" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:27.920428 3141 consensus_queue.cc:1035] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e364792037ca4b7aa9a667c89c9c7455" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:27.937901 3144 mvcc.cc:205] Tried to move back new op lower bound from 7315684753060286464 to 7315684752876544000. Current Snapshot: MvccSnapshot[applied={T|T < 7315684753060286464}]
W20260806 22:37:27.941259 2894 tablet.cc:2369] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:37:27.971442 2378 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:27.971911 2378 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:27.977654 2378 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1f2d398c6ab4d5fbf95a12d35531e35: Sending DeleteTablet for 3 replicas of tablet 18bd16ac0a844df087f25bda5c8dd65c
I20260806 22:37:27.979599 2580 tablet_service.cc:1514] Processing DeleteTablet for tablet 18bd16ac0a844df087f25bda5c8dd65c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:27 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:33474
I20260806 22:37:27.980151 2990 tablet_service.cc:1514] Processing DeleteTablet for tablet 18bd16ac0a844df087f25bda5c8dd65c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:27 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:56208
I20260806 22:37:27.980633 2801 tablet_service.cc:1514] Processing DeleteTablet for tablet 18bd16ac0a844df087f25bda5c8dd65c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:27 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:39098
I20260806 22:37:27.985453 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 2433
I20260806 22:37:28.000553 3158 tablet_replica.cc:330] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e: stopping tablet replica
I20260806 22:37:28.001639 3158 raft_consensus.cc:2229] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:28.004086 3158 raft_consensus.cc:2258] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:28.012311 3160 tablet_replica.cc:330] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455: stopping tablet replica
I20260806 22:37:28.012369 3158 ts_tablet_manager.cc:1905] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:28.013304 3160 raft_consensus.cc:2229] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:28.014814 3160 raft_consensus.cc:2258] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:28.014187 3157 tablet_replica.cc:330] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b: stopping tablet replica
I20260806 22:37:28.015698 3157 raft_consensus.cc:2229] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:37:28.016212 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:27.987083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33488 (local address 127.15.198.129:43549)
0806 22:37:27.987311 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:27.987317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:27.987351 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:37:27.987436 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:27.987443 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:27.987548 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:27.987685 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:27.998071 (+ 10386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:27.999642 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.001781 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.002593 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.002752 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:28.007365 (+ 4613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:28.007388 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:28.007412 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:28.007467 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:28.011994 (+ 4527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:28.013188 (+ 1194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:28.013199 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:28.013202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:28.013511 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:28.014086 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:28.014092 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:28.014096 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:28.014720 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:28.015143 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:28.015469 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 22:37:28.015779 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 22:37:28.017652 2674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:28.021497 3160 ts_tablet_manager.cc:1905] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:28.024264 3158 ts_tablet_manager.cc:1918] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:28.024535 3158 log.cc:1196] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/18bd16ac0a844df087f25bda5c8dd65c
I20260806 22:37:28.025267 3158 ts_tablet_manager.cc:1939] T 18bd16ac0a844df087f25bda5c8dd65c P e920a28a4bfc4a3591c63ae13603766e: Deleting consensus metadata
I20260806 22:37:28.028062 2359 catalog_manager.cc:4841] TS e920a28a4bfc4a3591c63ae13603766e (127.15.198.131:36469): tablet 18bd16ac0a844df087f25bda5c8dd65c (table test-table [id=ec1adb71c8ea43a2bd308583cd3d0c9d]) successfully deleted
I20260806 22:37:28.032863 3160 ts_tablet_manager.cc:1918] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:28.033224 3160 log.cc:1196] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/18bd16ac0a844df087f25bda5c8dd65c
I20260806 22:37:28.033973 3160 ts_tablet_manager.cc:1939] T 18bd16ac0a844df087f25bda5c8dd65c P e364792037ca4b7aa9a667c89c9c7455: Deleting consensus metadata
I20260806 22:37:28.036497 2358 catalog_manager.cc:4841] TS e364792037ca4b7aa9a667c89c9c7455 (127.15.198.130:36771): tablet 18bd16ac0a844df087f25bda5c8dd65c (table test-table [id=ec1adb71c8ea43a2bd308583cd3d0c9d]) successfully deleted
I20260806 22:37:28.169257 3157 raft_consensus.cc:2258] T 18bd16ac0a844df087f25bda5c8dd65c P 69625ba1ec6f4914880887a38221051b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:28.170401 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 2433
W20260806 22:37:28.211688 2360 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.198.129:43549 (error 108)
I20260806 22:37:28.211933 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 2691
I20260806 22:37:28.214119 3122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.213563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:43549 (local address 127.0.0.1:33502)
0806 22:37:28.213772 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:28.213928 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:43549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260806 22:37:28.214377 2360 catalog_manager.cc:4584] TS 69625ba1ec6f4914880887a38221051b (127.15.198.129:43549): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 18bd16ac0a844df087f25bda5c8dd65c: Network error: Client connection negotiation failed: client connection to 127.15.198.129:43549: connect: Connection refused (error 111)
I20260806 22:37:28.229516 3124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.213253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39104 (local address 127.15.198.130:36771)
0806 22:37:28.213516 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:28.213523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:28.214337 (+ 814us) server_negotiation.cc:407] Connection header received
0806 22:37:28.214451 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:28.214457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:28.214572 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:28.214694 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:28.216427 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.217263 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.218716 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.219333 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.219488 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:28.222543 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:28.222563 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:28.222585 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:28.222639 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:28.226033 (+ 3394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:28.226903 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:28.226912 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:28.226914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:28.227133 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:28.227577 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:28.227583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:28.227586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:28.228074 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:28.228433 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:28.228784 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 22:37:28.229081 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260806 22:37:28.230454 2886 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:28.268690 3122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.267985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:43549 (local address 127.0.0.1:33516)
0806 22:37:28.268284 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:28.268439 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:43549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260806 22:37:28.270149 3122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.269664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:43549 (local address 127.0.0.1:33522)
0806 22:37:28.269861 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:28.269962 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:43549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 22:37:28.331677 3122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.331009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:43549 (local address 127.0.0.1:33524)
0806 22:37:28.331293 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:28.331444 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:43549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 22:37:28.332894 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 2691
I20260806 22:37:28.333232 3122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.332732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:43549 (local address 127.0.0.1:33528)
0806 22:37:28.332902 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:28.333023 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:43549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 22:37:28.364462 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 2895
I20260806 22:37:28.381803 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.365546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56222 (local address 127.15.198.131:36469)
0806 22:37:28.366181 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:28.366186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:28.366207 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:37:28.366280 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:28.366284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:28.366373 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:28.366493 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:28.368307 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.369175 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.370961 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.371590 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.371704 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:28.374752 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:28.374772 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:28.374795 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:28.374837 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:28.378433 (+ 3596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:28.379532 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:28.379541 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:28.379544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:28.379703 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:28.380055 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:28.380059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:28.380060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:28.380493 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:28.380774 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:28.381110 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 22:37:28.381328 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":375,"threads_started":1}
I20260806 22:37:28.382722 3079 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:28.417596 3122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.416882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:43549 (local address 127.0.0.1:33534)
0806 22:37:28.417207 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:28.417337 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:43549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260806 22:37:28.419174 3122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.418560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:43549 (local address 127.0.0.1:33550)
0806 22:37:28.418853 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:28.418952 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:43549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260806 22:37:28.482432 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 2895
I20260806 22:37:28.509405 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 2328
I20260806 22:37:28.510938 2422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:28.644459 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 2328
I20260806 22:37:28.644543 3122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.643853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:43549 (local address 127.0.0.1:60836)
0806 22:37:28.644103 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:28.644231 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:43549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 22:37:28.686491 16154 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 22:37:29.033898 16154 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 22:37:29.034255 16154 mini_ranger.cc:66] Stopping Ranger...
I20260806 22:37:29.391703 16154 mini_ranger.cc:68] Stopped Ranger
2026-08-06 22:37:29.392 UTC [901] LOG: received smart shutdown request
2026-08-06 22:37:29.396 UTC [901] LOG: background worker "logical replication launcher" (PID 918) exited with exit code 1
2026-08-06 22:37:29.396 UTC [910] LOG: shutting down
2026-08-06 22:37:29.589 UTC [901] LOG: database system is shut down
2026-08-06T22:37:29Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (65262 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:37:29 dist-test-slave-h3ft krb5kdc[3180](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:37:29 dist-test-slave-h3ft krb5kdc[3180](info): set up 2 sockets
Aug 06 22:37:29 dist-test-slave-h3ft krb5kdc[3180](info): commencing operation
krb5kdc: starting...
W20260806 22:37:31.151091 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.350s user 0.004s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:37:31 dist-test-slave-h3ft krb5kdc[3180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:37:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:37:31.526959 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:41747
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:41747,127.15.198.189:33299,127.15.198.188:39461
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:31.976317 3210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:31.976785 3210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:31.976904 3210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:31.985553 3210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:31.985721 3210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:31.985775 3210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:31.985852 3210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:31.985904 3210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:31.997787 3210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:41747,127.15.198.189:33299,127.15.198.188:39461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:41747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:31.999773 3210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:32.001601 3210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:32.016062 3217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:32.016203 3218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:32.016093 3222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:32.017634 3210 server_base.cc:1034] running on GCE node
I20260806 22:37:32.018659 3210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:32.020649 3210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:32.021936 3210 hybrid_clock.cc:648] HybridClock initialized: now 1786055852021814 us; error 58 us; skew 500 ppm
I20260806 22:37:32.022341 3210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:32 dist-test-slave-h3ft krb5kdc[3180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055852, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:32.026660 3210 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:37:32.028666 3210 webserver.cc:460] Webserver started at http://127.15.198.190:45293/ using document root <none> and password file <none>
I20260806 22:37:32.029569 3210 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:32.029726 3210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:32.030122 3210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:32.034111 3210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "291130a0ca4f472abd979415fd8fd5e4"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-h3ft"
server_key: "7f1138079f6a0ad6034623ae5c3ae942"
server_key_iv: "54e92936dfd97ed918cc8c2d2ba6b09b"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.035275 3210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "291130a0ca4f472abd979415fd8fd5e4"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-h3ft"
server_key: "7f1138079f6a0ad6034623ae5c3ae942"
server_key_iv: "54e92936dfd97ed918cc8c2d2ba6b09b"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.042272 3210 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260806 22:37:32.047271 3234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:32.049057 3210 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:37:32.049297 3210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "291130a0ca4f472abd979415fd8fd5e4"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-h3ft"
server_key: "7f1138079f6a0ad6034623ae5c3ae942"
server_key_iv: "54e92936dfd97ed918cc8c2d2ba6b09b"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.049557 3210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:32.074331 3210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:32.079986 3210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:32.080382 3210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:32.104548 3210 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:41747
I20260806 22:37:32.104573 3312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:41747 every 8 connection(s)
I20260806 22:37:32.106462 3210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:32.108680 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 3210
I20260806 22:37:32.109017 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:37:32.110049 16154 external_mini_cluster.cc:1428] Setting key 553b122db54020fc296c09847610c368
I20260806 22:37:32.113267 3313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:32.127503 3313 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:37:32 dist-test-slave-h3ft krb5kdc[3180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055852, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.198.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.189@KRBTEST.COM" created.
I20260806 22:37:32.159305 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.132526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41747 (local address 127.0.0.1:53756)
0806 22:37:32.133877 (+ 1351us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.133946 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:32.136667 (+ 2721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:32.137921 (+ 1254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:32.137930 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:32.138863 (+ 933us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:32.139377 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:32.139403 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.141357 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.141366 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:32.142651 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:32.142666 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.144045 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.144056 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:32.144314 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:32.145337 (+ 1023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:32.145384 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:32.148946 (+ 3562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:32.153707 (+ 4761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:32.153723 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:32.153741 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:32.154252 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:32.154763 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:32.154768 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:32.154772 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:32.155051 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:32.156052 (+ 1001us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:32.156061 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:32.156324 (+ 263us) client_negotiation.cc:769] Sending connection context
0806 22:37:32.156621 (+ 297us) client_negotiation.cc:240] Negotiation successful
0806 22:37:32.157431 (+ 810us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":980,"mutex_wait_us":176,"thread_start_us":617,"threads_started":1}
I20260806 22:37:32.159322 3317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.134718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53756 (local address 127.15.198.190:41747)
0806 22:37:32.135686 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:32.135696 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:32.135777 (+ 81us) server_negotiation.cc:407] Connection header received
0806 22:37:32.137261 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:32.137268 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:32.137614 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:32.137900 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:32.139625 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.141190 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.142818 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.143631 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.144003 (+ 372us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:32.149283 (+ 5280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:32.149300 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:32.149327 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:32.149371 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:32.153522 (+ 4151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:32.154434 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:32.154444 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:32.154446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:32.154637 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:32.155177 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:32.155180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:32.155183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:32.155889 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:32.156140 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:32.156762 (+ 622us) server_negotiation.cc:299] Negotiation successful
0806 22:37:32.157289 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":708,"thread_start_us":485,"threads_started":1}
I20260806 22:37:32.164593 3313 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:37:32.166446 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.165697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:33299 (local address 127.0.0.1:52484)
0806 22:37:32.166018 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.166212 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:33299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260806 22:37:32.167167 3238 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.198.189:33299: connect: Connection refused (error 111)
I20260806 22:37:32.170117 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.169424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:33299 (local address 127.0.0.1:52500)
0806 22:37:32.169751 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.169870 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:33299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
W20260806 22:37:32.170498 3313 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.189:33299: Network error: Client connection negotiation failed: client connection to 127.15.198.189:33299: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.189.keytab.
Entry for principal kudu/127.15.198.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.189.keytab.
I20260806 22:37:32.200450 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.189:33299
--webserver_interface=127.15.198.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.189.keytab
--principal=kudu/127.15.198.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:41747,127.15.198.189:33299,127.15.198.188:39461
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:37:32.220155 3313 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } attempt: 1
I20260806 22:37:32.221909 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.221302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:33299 (local address 127.0.0.1:52516)
0806 22:37:32.221574 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.221731 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:33299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 22:37:32.223255 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.222759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:33299 (local address 127.0.0.1:52526)
0806 22:37:32.222933 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.223073 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:33299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 22:37:32.223477 3313 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.189:33299: Network error: Client connection negotiation failed: client connection to 127.15.198.189:33299: connect: Connection refused (error 111)
I20260806 22:37:32.291764 3313 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } attempt: 2
I20260806 22:37:32.293433 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.292782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:33299 (local address 127.0.0.1:52528)
0806 22:37:32.293104 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.293228 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:33299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 22:37:32.294796 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.294331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:33299 (local address 127.0.0.1:52544)
0806 22:37:32.294555 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.294635 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:33299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 22:37:32.295054 3313 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.189:33299: Network error: Client connection negotiation failed: client connection to 127.15.198.189:33299: connect: Connection refused (error 111)
I20260806 22:37:32.386372 3313 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } attempt: 3
I20260806 22:37:32.388103 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.387495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:33299 (local address 127.0.0.1:52552)
0806 22:37:32.387759 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.387915 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:33299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 22:37:32.389739 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.389173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:33299 (local address 127.0.0.1:52558)
0806 22:37:32.389406 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.389514 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:33299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260806 22:37:32.389997 3313 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.189:33299: Network error: Client connection negotiation failed: client connection to 127.15.198.189:33299: connect: Connection refused (error 111)
I20260806 22:37:32.533334 3313 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } attempt: 4
I20260806 22:37:32.535089 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.534307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:33299 (local address 127.0.0.1:52572)
0806 22:37:32.534724 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.534836 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:33299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217}
I20260806 22:37:32.536345 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.535924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:33299 (local address 127.0.0.1:52582)
0806 22:37:32.536100 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.536200 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:33299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 22:37:32.536628 3313 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.189:33299: Network error: Client connection negotiation failed: client connection to 127.15.198.189:33299: connect: Connection refused (error 111)
W20260806 22:37:32.639235 3322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:32.639672 3322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:32.639851 3322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:32.648458 3322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:32.648630 3322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:32.648749 3322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:32.648864 3322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:32.648936 3322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:32.660771 3322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:41747,127.15.198.189:33299,127.15.198.188:39461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.189.keytab
--principal=kudu/127.15.198.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.189:33299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.198.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:32.662812 3322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:32.664687 3322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:32.680078 3330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:32.680130 3333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:32.680155 3335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:32.681169 3322 server_base.cc:1034] running on GCE node
I20260806 22:37:32.682348 3322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:32.684336 3322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:32.685613 3322 hybrid_clock.cc:648] HybridClock initialized: now 1786055852685536 us; error 69 us; skew 500 ppm
I20260806 22:37:32.686077 3322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:32 dist-test-slave-h3ft krb5kdc[3180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055852, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:32.696084 3322 init.cc:377] Logged in from keytab as kudu/127.15.198.189@KRBTEST.COM (short username kudu)
I20260806 22:37:32.698163 3322 webserver.cc:460] Webserver started at http://127.15.198.189:36697/ using document root <none> and password file <none>
I20260806 22:37:32.699191 3322 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:32.699357 3322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:32.699731 3322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:32.703857 3322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-1/data/instance:
uuid: "17501eba618745a891daef47a8aeef6c"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-h3ft"
server_key: "655e2f9e003df85cc7ba3f6a3cf29b13"
server_key_iv: "d223eb32ba14aa16698a762db35fc15d"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.704989 3322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-1/wal/instance:
uuid: "17501eba618745a891daef47a8aeef6c"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-h3ft"
server_key: "655e2f9e003df85cc7ba3f6a3cf29b13"
server_key_iv: "d223eb32ba14aa16698a762db35fc15d"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.711946 3322 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260806 22:37:32.716564 3345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:32.718330 3322 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260806 22:37:32.718573 3322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-1/wal
uuid: "17501eba618745a891daef47a8aeef6c"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-h3ft"
server_key: "655e2f9e003df85cc7ba3f6a3cf29b13"
server_key_iv: "d223eb32ba14aa16698a762db35fc15d"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.718858 3322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:32.764253 3322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:32.770422 3322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:32.770871 3322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:32.796002 3322 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.189:33299
I20260806 22:37:32.795981 3424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.189:33299 every 8 connection(s)
I20260806 22:37:32.797958 3322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-1/data/info.pb
I20260806 22:37:32.803027 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 3322
I20260806 22:37:32.803296 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-1/wal/instance
I20260806 22:37:32.803362 3425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:32.804000 16154 external_mini_cluster.cc:1428] Setting key 4f7405b42a17d276ed90154016d8b139
I20260806 22:37:32.816087 3425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:37:32.835948 3313 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } attempt: 5
Aug 06 22:37:32 dist-test-slave-h3ft krb5kdc[3180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055852, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.189@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.198.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.188@KRBTEST.COM" created.
I20260806 22:37:32.847608 3431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.820876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53772 (local address 127.15.198.190:41747)
0806 22:37:32.821511 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:32.821517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:32.823323 (+ 1806us) server_negotiation.cc:407] Connection header received
0806 22:37:32.825926 (+ 2603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:32.825936 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:32.826063 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:32.826214 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:32.828246 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.829723 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.831904 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.832797 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.833025 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:32.837853 (+ 4828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:32.837860 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:32.837863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:32.837916 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:32.842188 (+ 4272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:32.843550 (+ 1362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:32.843556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:32.843560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:32.843726 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:32.844319 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:32.844324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:32.844327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:32.844768 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:32.844948 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:32.846513 (+ 1565us) server_negotiation.cc:299] Negotiation successful
0806 22:37:32.846850 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":364,"threads_started":1}
I20260806 22:37:32.849138 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.821990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41747 (local address 127.0.0.1:53772)
0806 22:37:32.823101 (+ 1111us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.823161 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:32.825499 (+ 2338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:32.826545 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:32.826555 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:32.827284 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:32.828019 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:32.828041 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.829959 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.829964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:32.831705 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:32.831721 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.833129 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.833135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:32.833381 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:32.834364 (+ 983us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:32.834393 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:32.837565 (+ 3172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:32.842359 (+ 4794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:32.842370 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:32.842393 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:32.843347 (+ 954us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:32.843873 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:32.843879 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:32.843883 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:32.844160 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:32.844961 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:32.844979 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:32.845756 (+ 777us) client_negotiation.cc:769] Sending connection context
0806 22:37:32.846167 (+ 411us) client_negotiation.cc:240] Negotiation successful
0806 22:37:32.846854 (+ 687us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":459,"threads_started":1}
Aug 06 22:37:32 dist-test-slave-h3ft krb5kdc[3180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055852, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for kudu/127.15.198.189@KRBTEST.COM
I20260806 22:37:32.853426 3425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:37:32.863245 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.836809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:33299 (local address 127.0.0.1:52592)
0806 22:37:32.837082 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.837105 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:32.837239 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:32.839429 (+ 2190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:32.839439 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:32.840175 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:32.840563 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:32.840580 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.842900 (+ 2320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.842906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:32.844417 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:32.844432 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.846104 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.846115 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:32.846533 (+ 418us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:32.847515 (+ 982us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:32.847566 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:32.850860 (+ 3294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:32.859646 (+ 8786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:32.859653 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:32.859657 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:32.860328 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:32.860896 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:32.860901 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:32.860904 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:32.861087 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:32.862005 (+ 918us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:32.862010 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:32.862207 (+ 197us) client_negotiation.cc:769] Sending connection context
0806 22:37:32.862341 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 22:37:32.862765 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 22:37:32.865000 3433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.837667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52592 (local address 127.15.198.189:33299)
0806 22:37:32.838685 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:32.838697 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:32.838745 (+ 48us) server_negotiation.cc:407] Connection header received
0806 22:37:32.838822 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:32.838829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:32.839163 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:32.839385 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:32.840820 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.842689 (+ 1869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.844595 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.845871 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.846051 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:32.851229 (+ 5178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:32.851247 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:32.851267 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:32.851312 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:32.859426 (+ 8114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:32.860550 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:32.860560 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:32.860565 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:32.860757 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:32.861228 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:32.861233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:32.861236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:32.861852 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:32.863853 (+ 2001us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:32.864084 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 22:37:32.864426 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":777,"thread_start_us":579,"threads_started":1}
I20260806 22:37:32.869771 3313 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:37:32.871519 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.870819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47148)
0806 22:37:32.871163 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.871282 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260806 22:37:32.872998 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.872432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47156)
0806 22:37:32.872705 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.872827 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
W20260806 22:37:32.873243 3313 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:39461: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Aug 06 22:37:32 dist-test-slave-h3ft krb5kdc[3180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055852, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.189@KRBTEST.COM for kudu/127.15.198.189@KRBTEST.COM
I20260806 22:37:32.882939 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.855089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:33299 (local address 127.0.0.1:52600)
0806 22:37:32.859262 (+ 4173us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.859289 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:32.859441 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:32.867116 (+ 7675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:32.867124 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:32.867771 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:32.868058 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:32.868075 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.869831 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.869838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:32.871295 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:32.871311 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.873448 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.873454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:32.873683 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:32.874397 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:32.874417 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:32.876968 (+ 2551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:32.880343 (+ 3375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:32.880349 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:32.880354 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:32.880737 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:32.881129 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:32.881134 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:32.881138 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:32.881297 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:32.881855 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:32.881860 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:32.882024 (+ 164us) client_negotiation.cc:769] Sending connection context
0806 22:37:32.882165 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 22:37:32.882502 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3965}
I20260806 22:37:32.883054 3433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.855089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52600 (local address 127.15.198.189:33299)
0806 22:37:32.866684 (+ 11595us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:32.866693 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:32.866719 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:37:32.866786 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:32.866793 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:32.866921 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:32.868217 (+ 1296us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:32.868386 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.869666 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.871470 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.873283 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.873415 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:32.877193 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:32.877199 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:32.877202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:32.877236 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:32.880182 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:32.880871 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:32.880876 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:32.880878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:32.880999 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:32.881410 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:32.881415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:32.881418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:32.881739 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:32.881905 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:32.882270 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 22:37:32.882676 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":154,"server-negotiator.queue_time_us":11449,"thread_start_us":14545,"threads_started":1}
I20260806 22:37:32.884353 3425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:37:32.885723 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.885153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47170)
0806 22:37:32.885386 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.885552 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260806 22:37:32.886094 3350 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
I20260806 22:37:32.888006 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.887637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47182)
0806 22:37:32.887807 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.887886 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 22:37:32.888363 3425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:39461: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.188.keytab.
Entry for principal kudu/127.15.198.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.188.keytab.
I20260806 22:37:32.901377 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.188:39461
--webserver_interface=127.15.198.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.188.keytab
--principal=kudu/127.15.198.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:41747,127.15.198.189:33299,127.15.198.188:39461
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:37:32.924619 3313 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } attempt: 1
I20260806 22:37:32.926396 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.925631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47184)
0806 22:37:32.925946 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.926106 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 22:37:32.927847 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.927335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47196)
0806 22:37:32.927540 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.927651 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 22:37:32.928121 3313 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:39461: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
I20260806 22:37:32.937826 3425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } attempt: 1
I20260806 22:37:32.939502 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.938860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47198)
0806 22:37:32.939156 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.939287 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 22:37:32.940726 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.940292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47206)
0806 22:37:32.940490 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.940574 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 22:37:32.940997 3425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:39461: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
I20260806 22:37:32.996450 3313 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } attempt: 2
I20260806 22:37:32.998706 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.997516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47210)
0806 22:37:32.998149 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.998347 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 22:37:33.000267 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.999728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47218)
0806 22:37:33.000008 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:33.000121 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260806 22:37:33.000599 3313 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:39461: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
I20260806 22:37:33.009268 3425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } attempt: 2
I20260806 22:37:33.010824 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.010239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47220)
0806 22:37:33.010501 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:33.010624 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 22:37:33.011988 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.011608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47230)
0806 22:37:33.011768 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:33.011852 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 22:37:33.012228 3425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:39461: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
I20260806 22:37:33.103591 3425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } attempt: 3
I20260806 22:37:33.105551 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.104718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47236)
0806 22:37:33.105119 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:33.105277 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260806 22:37:33.106866 3313 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } attempt: 3
I20260806 22:37:33.107352 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.106493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47242)
0806 22:37:33.106838 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:33.106966 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184}
W20260806 22:37:33.107704 3425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:39461: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
I20260806 22:37:33.108222 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.107604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47248)
0806 22:37:33.107904 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:33.108008 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
I20260806 22:37:33.109633 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.109089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47262)
0806 22:37:33.109364 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:33.109468 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260806 22:37:33.109927 3313 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:39461: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
I20260806 22:37:33.251032 3425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } attempt: 4
I20260806 22:37:33.253010 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.252225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47270)
0806 22:37:33.252550 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:33.252710 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260806 22:37:33.254563 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.253988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47276)
0806 22:37:33.254193 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:33.254341 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 22:37:33.254840 3425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:39461: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
I20260806 22:37:33.287220 3313 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } attempt: 4
I20260806 22:37:33.288879 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.288140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47282)
0806 22:37:33.288443 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:33.288605 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 22:37:33.290217 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.289715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47286)
0806 22:37:33.289909 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:33.290019 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 22:37:33.290482 3313 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:39461: Network error: Client connection negotiation failed: client connection to 127.15.198.188:39461: connect: Connection refused (error 111)
W20260806 22:37:33.354293 3439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:33.354698 3439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:33.354799 3439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:33.363143 3439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:33.363304 3439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:33.363361 3439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:33.363433 3439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:33.363483 3439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:33.375373 3439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:41747,127.15.198.189:33299,127.15.198.188:39461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.188.keytab
--principal=kudu/127.15.198.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.188:39461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.198.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:33.377210 3439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:33.379076 3439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:33.393661 3447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:33.393618 3445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:33.395071 3450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:33.395498 3439 server_base.cc:1034] running on GCE node
I20260806 22:37:33.396633 3439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:33.398587 3439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:33.399921 3439 hybrid_clock.cc:648] HybridClock initialized: now 1786055853399786 us; error 51 us; skew 500 ppm
I20260806 22:37:33.400398 3439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:33 dist-test-slave-h3ft krb5kdc[3180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055853, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:33.404520 3439 init.cc:377] Logged in from keytab as kudu/127.15.198.188@KRBTEST.COM (short username kudu)
I20260806 22:37:33.406550 3439 webserver.cc:460] Webserver started at http://127.15.198.188:34563/ using document root <none> and password file <none>
I20260806 22:37:33.407627 3439 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:33.407783 3439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:33.408187 3439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:33.412330 3439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-2/data/instance:
uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f"
format_stamp: "Formatted at 2026-08-06 22:37:33 on dist-test-slave-h3ft"
server_key: "e8e61f1b678be40b391660eda87d60c3"
server_key_iv: "15fd6177744b32a51e467bffa65e7244"
server_key_version: "encryptionkey@0"
I20260806 22:37:33.413550 3439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-2/wal/instance:
uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f"
format_stamp: "Formatted at 2026-08-06 22:37:33 on dist-test-slave-h3ft"
server_key: "e8e61f1b678be40b391660eda87d60c3"
server_key_iv: "15fd6177744b32a51e467bffa65e7244"
server_key_version: "encryptionkey@0"
I20260806 22:37:33.420816 3439 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260806 22:37:33.425616 3459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:33.427673 3439 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:37:33.427904 3439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-2/wal
uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f"
format_stamp: "Formatted at 2026-08-06 22:37:33 on dist-test-slave-h3ft"
server_key: "e8e61f1b678be40b391660eda87d60c3"
server_key_iv: "15fd6177744b32a51e467bffa65e7244"
server_key_version: "encryptionkey@0"
I20260806 22:37:33.428221 3439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:33.452191 3439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:33.458202 3439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:33.458583 3439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:33.482728 3439 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.188:39461
I20260806 22:37:33.482762 3537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.188:39461 every 8 connection(s)
I20260806 22:37:33.484570 3439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-2/data/info.pb
I20260806 22:37:33.489754 3538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:33.494837 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 3439
I20260806 22:37:33.495126 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055595907839-16154-0/minicluster-data/master-2/wal/instance
I20260806 22:37:33.495816 16154 external_mini_cluster.cc:1428] Setting key c2cc35314da1ce21133c4ac782574ae9
I20260806 22:37:33.499868 3538 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:37:33 dist-test-slave-h3ft krb5kdc[3180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.188@KRBTEST.COM
Aug 06 22:37:33 dist-test-slave-h3ft krb5kdc[3180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055853, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.188@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:37:33.525913 3542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.505152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53782 (local address 127.15.198.190:41747)
0806 22:37:33.505855 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:33.505862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:33.506517 (+ 655us) server_negotiation.cc:407] Connection header received
0806 22:37:33.506740 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:33.506744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:33.506828 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:33.506948 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:33.508820 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.510092 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.512502 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.513216 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.513401 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:33.518403 (+ 5002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:33.518410 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:33.518414 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:33.518479 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:33.521853 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.522795 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.522800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.522803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.522966 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.523591 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.523597 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.523600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.523977 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:33.524180 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:33.525057 (+ 877us) server_negotiation.cc:299] Negotiation successful
0806 22:37:33.525350 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":315,"threads_started":1}
I20260806 22:37:33.527336 3545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.505136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41747 (local address 127.0.0.1:53782)
0806 22:37:33.506316 (+ 1180us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:33.506390 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:33.506587 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:33.507084 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:33.507092 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:33.508123 (+ 1031us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:33.508629 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:33.508640 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.510345 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.510350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:33.512330 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:33.512344 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.513522 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.513528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:33.513684 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:33.514398 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:33.514435 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:33.518120 (+ 3685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:33.522062 (+ 3942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:33.522078 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:33.522096 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:33.522623 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:33.523154 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:33.523158 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:33.523162 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:33.523434 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:33.524123 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:33.524134 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:33.524596 (+ 462us) client_negotiation.cc:769] Sending connection context
0806 22:37:33.524918 (+ 322us) client_negotiation.cc:240] Negotiation successful
0806 22:37:33.525658 (+ 740us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":855,"mutex_wait_us":82,"thread_start_us":650,"threads_started":1}
I20260806 22:37:33.527372 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.498477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47288 (local address 127.15.198.188:39461)
0806 22:37:33.499575 (+ 1098us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:33.499590 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:33.499643 (+ 53us) server_negotiation.cc:407] Connection header received
0806 22:37:33.502246 (+ 2603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:33.502281 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:33.502916 (+ 635us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:33.503471 (+ 555us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:33.505733 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.508242 (+ 2509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.510481 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.512069 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.512330 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:33.518137 (+ 5807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:33.518150 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:33.518170 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:33.518213 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:33.521940 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.522873 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.522889 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.522892 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.523192 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.523711 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.523718 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.523722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.524617 (+ 895us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:33.525014 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:33.525411 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 22:37:33.525806 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":838,"thread_start_us":569,"threads_started":1}
I20260806 22:37:33.529767 3538 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:37:33 dist-test-slave-h3ft krb5kdc[3180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055853, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.188@KRBTEST.COM for kudu/127.15.198.189@KRBTEST.COM
Aug 06 22:37:33 dist-test-slave-h3ft krb5kdc[3180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:37:33.550825 3545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.530854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:33299 (local address 127.0.0.1:52604)
0806 22:37:33.531110 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:33.531131 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:33.531270 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:33.532986 (+ 1716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:33.532992 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:33.533622 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:33.533919 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:33.533933 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.535770 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.535775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:33.537419 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:33.537445 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.538749 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.538757 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:33.539073 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:33.540108 (+ 1035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:33.540139 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:33.543341 (+ 3202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:33.547471 (+ 4130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:33.547478 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:33.547482 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:33.548170 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:33.548725 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:33.548731 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:33.548735 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:33.548929 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:33.549607 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:33.549613 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:33.549817 (+ 204us) client_negotiation.cc:769] Sending connection context
0806 22:37:33.549981 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 22:37:33.550376 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 22:37:33.550896 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.531447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52604 (local address 127.15.198.189:33299)
0806 22:37:33.532528 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:33.532537 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:33.532599 (+ 62us) server_negotiation.cc:407] Connection header received
0806 22:37:33.532682 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:33.532687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:33.532779 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:33.532940 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:33.534133 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.535479 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.537661 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.538451 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.538625 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:33.543713 (+ 5088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:33.543720 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:33.543725 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:33.543772 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:33.547258 (+ 3486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.548350 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.548356 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.548359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.548545 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.549100 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.549105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.549108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.549463 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:33.549650 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:33.550122 (+ 472us) server_negotiation.cc:299] Negotiation successful
0806 22:37:33.550398 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":920,"thread_start_us":727,"threads_started":1}
I20260806 22:37:33.551810 3542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.531971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53794 (local address 127.15.198.190:41747)
0806 22:37:33.532212 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:33.532223 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:33.532249 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:37:33.532346 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:33.532352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:33.532480 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:33.532635 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:33.534565 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.535806 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.537874 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.538642 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.538785 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:33.544707 (+ 5922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:33.544714 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:33.544718 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:33.544791 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:33.548406 (+ 3615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.549263 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.549270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.549273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.549439 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.549887 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.549891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.549893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.550209 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:33.550538 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:33.550848 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 22:37:33.551432 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 22:37:33.552526 3538 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:37:33.554170 3425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } attempt: 5
Aug 06 22:37:33 dist-test-slave-h3ft krb5kdc[3180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055853, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.188@KRBTEST.COM for kudu/127.15.198.188@KRBTEST.COM
Aug 06 22:37:33 dist-test-slave-h3ft krb5kdc[3180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055852, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.189@KRBTEST.COM for kudu/127.15.198.188@KRBTEST.COM
Aug 06 22:37:33 dist-test-slave-h3ft krb5kdc[3180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.189@KRBTEST.COM
I20260806 22:37:33.567756 3313 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } attempt: 5
I20260806 22:37:33.575260 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.554803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47302)
0806 22:37:33.555099 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:33.555119 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:33.555235 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:33.556690 (+ 1455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:33.556696 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:33.557367 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:33.557611 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:33.557625 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.559522 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.559528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:33.560939 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:33.560959 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.561987 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.561994 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:33.562238 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:33.562912 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:33.562931 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:33.565441 (+ 2510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:33.571067 (+ 5626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:33.571076 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:33.571081 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:33.571559 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:33.572765 (+ 1206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:33.572773 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:33.572779 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:33.573000 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:33.573957 (+ 957us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:33.573963 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:33.574176 (+ 213us) client_negotiation.cc:769] Sending connection context
0806 22:37:33.574329 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 22:37:33.574733 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143}
I20260806 22:37:33.575233 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.553729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47292 (local address 127.15.198.188:39461)
0806 22:37:33.553931 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:33.553938 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:33.554770 (+ 832us) server_negotiation.cc:407] Connection header received
0806 22:37:33.554990 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:33.554995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:33.555112 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:33.555225 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:33.556710 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.557888 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.559202 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.559811 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.559962 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:33.564758 (+ 4796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:33.564767 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:33.564773 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:33.564821 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:33.570278 (+ 5457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.572679 (+ 2401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.572692 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.572696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.572867 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.573323 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.573328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.573331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.573807 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:33.573965 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:33.574462 (+ 497us) server_negotiation.cc:299] Negotiation successful
0806 22:37:33.574735 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":48,"spinlock_wait_cycles":896}
I20260806 22:37:33.576435 3547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.555233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47302 (local address 127.15.198.188:39461)
0806 22:37:33.556249 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:33.556255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:33.556281 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:37:33.556340 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:33.556344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:33.556411 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:33.556616 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:33.557803 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.559361 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.561118 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.561784 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.561962 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:33.565740 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:33.565745 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:33.565748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:33.565779 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:33.570826 (+ 5047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.571694 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.571703 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.571706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.571925 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.573160 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.573164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.573167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.573807 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:33.575564 (+ 1757us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:33.575731 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 22:37:33.575968 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":867,"spinlock_wait_cycles":43776,"thread_start_us":491,"threads_started":1}
I20260806 22:37:33.577083 3545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.553489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47292)
0806 22:37:33.554691 (+ 1202us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:33.554715 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:33.554861 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:33.555906 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:33.555909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:33.556350 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:33.556534 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:33.556543 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.558021 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.558025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:33.559047 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:33.559064 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.559977 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.559983 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:33.560242 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:33.561128 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:33.561157 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:33.564508 (+ 3351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:33.571739 (+ 7231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:33.571744 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:33.571748 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:33.572555 (+ 807us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:33.573017 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:33.573022 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:33.573026 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:33.573211 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:33.574032 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:33.574036 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:33.574214 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 22:37:33.576184 (+ 1970us) client_negotiation.cc:240] Negotiation successful
0806 22:37:33.576816 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1062,"mutex_wait_us":1}
I20260806 22:37:33.577479 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.554153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52616 (local address 127.15.198.189:33299)
0806 22:37:33.554341 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:33.554347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:33.554368 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:37:33.554466 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:33.554471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:33.554566 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:33.554677 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:33.556777 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.557662 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.559794 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.560551 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.560693 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:33.566910 (+ 6217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:33.566919 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:33.566927 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:33.566977 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:33.572280 (+ 5303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.573534 (+ 1254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.573540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.573544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.573727 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.574343 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.574349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.574352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.574818 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:33.575242 (+ 424us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:33.576782 (+ 1540us) server_negotiation.cc:299] Negotiation successful
0806 22:37:33.577111 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 22:37:33 dist-test-slave-h3ft krb5kdc[3180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055852, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for kudu/127.15.198.188@KRBTEST.COM
I20260806 22:37:33.584250 3538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f: Bootstrap starting.
I20260806 22:37:33.584283 3425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c: Bootstrap starting.
I20260806 22:37:33.589285 3425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:33.589303 3538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:33.590874 3538 log.cc:824] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:33.592322 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.568640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:39461 (local address 127.0.0.1:47304)
0806 22:37:33.569023 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:33.569055 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:33.569220 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:33.570981 (+ 1761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:33.570989 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:33.571687 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:33.571914 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:33.571927 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.574296 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.574307 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:33.576156 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:33.576196 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.578157 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.578166 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:33.578419 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:33.579340 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:33.579372 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:33.582809 (+ 3437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:33.588083 (+ 5274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:33.588089 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:33.588094 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:33.588850 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:33.589548 (+ 698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:33.589556 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:33.589563 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:33.589818 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:33.590691 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:33.590697 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:33.590967 (+ 270us) client_negotiation.cc:769] Sending connection context
0806 22:37:33.591326 (+ 359us) client_negotiation.cc:240] Negotiation successful
0806 22:37:33.591757 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 22:37:33.592533 3548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.569305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47304 (local address 127.15.198.188:39461)
0806 22:37:33.570590 (+ 1285us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:33.570595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:33.570619 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:37:33.570675 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:33.570680 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:33.570752 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:33.570896 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:33.572146 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.573774 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.577186 (+ 3412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.577954 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.578110 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:33.583151 (+ 5041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:33.583162 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:33.583173 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:33.583226 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:33.587856 (+ 4630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.589072 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.589082 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.589089 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.589353 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.590019 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.590025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.590028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.590519 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:33.590722 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:33.591479 (+ 757us) server_negotiation.cc:299] Negotiation successful
0806 22:37:33.591868 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1130,"thread_start_us":892,"threads_started":1}
I20260806 22:37:33.590901 3425 log.cc:824] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:33.597083 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:37:33.597507 3538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f: No bootstrap required, opened a new log
I20260806 22:37:33.600283 3313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4: Bootstrap starting.
I20260806 22:37:33.605841 3313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:33.607846 3313 log.cc:824] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:33.611675 3425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c: No bootstrap required, opened a new log
I20260806 22:37:33.613085 3538 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } }
I20260806 22:37:33.614043 3538 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:33.614271 3538 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3ecbf9a9d9e4efc9e3d58421bfb1a8f, State: Initialized, Role: FOLLOWER
I20260806 22:37:33.615718 3538 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } }
I20260806 22:37:33.629861 3561 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } } }
I20260806 22:37:33.630487 3561 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:37:33.632310 3313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4: No bootstrap required, opened a new log
I20260806 22:37:33.633018 3538 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:33.633263 3425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } }
I20260806 22:37:33.633944 3425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:33.634104 3425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17501eba618745a891daef47a8aeef6c, State: Initialized, Role: FOLLOWER
I20260806 22:37:33.635044 3425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } }
I20260806 22:37:33.642328 3569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } } }
I20260806 22:37:33.642887 3569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:37:33.645551 3425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:33.647562 3313 raft_consensus.cc:348] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } }
I20260806 22:37:33.648322 3313 raft_consensus.cc:374] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:33.648514 3313 raft_consensus.cc:729] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 291130a0ca4f472abd979415fd8fd5e4, State: Initialized, Role: FOLLOWER
I20260806 22:37:33.650964 3313 consensus_queue.cc:260] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } }
I20260806 22:37:33.662201 3313 sys_catalog.cc:564] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:33.658901 3572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } } }
I20260806 22:37:33.665225 3572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:37:33.683095 3572 raft_consensus.cc:482] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:33.683531 3572 raft_consensus.cc:504] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } }
I20260806 22:37:33.685696 3572 leader_election.cc:288] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17501eba618745a891daef47a8aeef6c (127.15.198.189:33299), f3ecbf9a9d9e4efc9e3d58421bfb1a8f (127.15.198.188:39461)
I20260806 22:37:33.688334 3385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "291130a0ca4f472abd979415fd8fd5e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17501eba618745a891daef47a8aeef6c" is_pre_election: true
I20260806 22:37:33.689164 3385 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 291130a0ca4f472abd979415fd8fd5e4 in term 0.
I20260806 22:37:33.690158 3501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "291130a0ca4f472abd979415fd8fd5e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" is_pre_election: true
I20260806 22:37:33.691747 3238 leader_election.cc:302] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17501eba618745a891daef47a8aeef6c, 291130a0ca4f472abd979415fd8fd5e4; no voters:
I20260806 22:37:33.690977 3501 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 291130a0ca4f472abd979415fd8fd5e4 in term 0.
I20260806 22:37:33.693686 3572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:33.693885 3572 raft_consensus.cc:482] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:33.693988 3572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:33.698690 3572 raft_consensus.cc:504] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } }
I20260806 22:37:33.700687 3385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "291130a0ca4f472abd979415fd8fd5e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17501eba618745a891daef47a8aeef6c"
I20260806 22:37:33.701105 3385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:33.705210 3385 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 291130a0ca4f472abd979415fd8fd5e4 in term 1.
I20260806 22:37:33.705905 3238 leader_election.cc:302] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17501eba618745a891daef47a8aeef6c, 291130a0ca4f472abd979415fd8fd5e4; no voters:
W20260806 22:37:33.711688 3468 tablet.cc:2369] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:37:33.716737 3501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "291130a0ca4f472abd979415fd8fd5e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f"
I20260806 22:37:33.717154 3501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:33.722669 3501 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 291130a0ca4f472abd979415fd8fd5e4 in term 1.
I20260806 22:37:33.724738 3572 leader_election.cc:288] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [CANDIDATE]: Term 1 election: Requested vote from peers 17501eba618745a891daef47a8aeef6c (127.15.198.189:33299), f3ecbf9a9d9e4efc9e3d58421bfb1a8f (127.15.198.188:39461)
I20260806 22:37:33.725167 3572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:33.725414 3572 raft_consensus.cc:686] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [term 1 LEADER]: Becoming Leader. State: Replica: 291130a0ca4f472abd979415fd8fd5e4, State: Running, Role: LEADER
I20260806 22:37:33.726337 3572 consensus_queue.cc:237] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } }
I20260806 22:37:33.728786 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.659458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52622 (local address 127.15.198.189:33299)
0806 22:37:33.660251 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:33.660262 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:33.666398 (+ 6136us) server_negotiation.cc:407] Connection header received
0806 22:37:33.685426 (+ 19028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:33.685439 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:33.685600 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:33.685718 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:33.694862 (+ 9144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.696961 (+ 2099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.706964 (+ 10003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.708417 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.708629 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:33.715858 (+ 7229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:33.715867 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:33.715871 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:33.715951 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:33.723107 (+ 7156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.724440 (+ 1333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.724446 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.724450 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.724703 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.725468 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.725475 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.725478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.726097 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:33.726429 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:33.727997 (+ 1568us) server_negotiation.cc:299] Negotiation successful
0806 22:37:33.728308 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563}
I20260806 22:37:33.732518 3542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.652871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53808 (local address 127.15.198.190:41747)
0806 22:37:33.653187 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:33.653198 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:33.663793 (+ 10595us) server_negotiation.cc:407] Connection header received
0806 22:37:33.666768 (+ 2975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:33.666780 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:33.666937 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:33.667074 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:33.690979 (+ 23905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.693759 (+ 2780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.703943 (+ 10184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.705459 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.705694 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:33.715654 (+ 9960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:33.715665 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:33.715671 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:33.715740 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:33.725665 (+ 9925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.726683 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.726691 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.726697 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.726907 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.727420 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.727426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.727429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.727886 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:33.728168 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:33.728867 (+ 699us) server_negotiation.cc:299] Negotiation successful
0806 22:37:33.729172 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260806 22:37:33.750687 3593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 291130a0ca4f472abd979415fd8fd5e4. Latest consensus state: current_term: 1 leader_uuid: "291130a0ca4f472abd979415fd8fd5e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } } }
I20260806 22:37:33.751564 3593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:33.766566 3548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:33.664759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47320 (local address 127.15.198.188:39461)
0806 22:37:33.665074 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:33.665086 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:33.685982 (+ 20896us) server_negotiation.cc:407] Connection header received
0806 22:37:33.686099 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:33.686106 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:33.686243 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:33.686391 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:33.736566 (+ 50175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.738308 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.741558 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:33.742691 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:33.742859 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:33.752327 (+ 9468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:33.752340 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:33.752351 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:33.752453 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:33.758298 (+ 5845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.759947 (+ 1649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.759960 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.759967 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.760219 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:33.762555 (+ 2336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:33.762567 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:33.762573 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:33.763330 (+ 757us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:33.763849 (+ 519us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:33.765512 (+ 1663us) server_negotiation.cc:299] Negotiation successful
0806 22:37:33.765952 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 22:37:33.772982 3605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260806 22:37:33.774726 3356 tablet.cc:2369] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:37:33.820175 3605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260806 22:37:33.834810 3247 tablet.cc:2369] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:37:33.848235 3439 master_runner.cc:186] Error getting master registration for 127.15.198.189:33299: OK, instance_id { permanent_uuid: "17501eba618745a891daef47a8aeef6c" instance_seqno: 1786055852788788 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:37:33.866359 3439 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:37:33.781939 3210 master_runner.cc:186] Error getting master registration for 127.15.198.189:33299: OK, instance_id { permanent_uuid: "17501eba618745a891daef47a8aeef6c" instance_seqno: 1786055852788788 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:37:33.866941 3210 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:37:33.873797 3385 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c [term 1 FOLLOWER]: Refusing update from remote peer 291130a0ca4f472abd979415fd8fd5e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:33.875034 3593 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:33.876072 3501 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f [term 1 FOLLOWER]: Refusing update from remote peer 291130a0ca4f472abd979415fd8fd5e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:33.879390 3593 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:37:33.882545 3620 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:37:33.882735 3620 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:37:33.909384 3561 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 291130a0ca4f472abd979415fd8fd5e4. Latest consensus state: current_term: 1 leader_uuid: "291130a0ca4f472abd979415fd8fd5e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } } }
I20260806 22:37:33.909832 3561 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:37:33.912509 3569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 291130a0ca4f472abd979415fd8fd5e4. Latest consensus state: current_term: 1 leader_uuid: "291130a0ca4f472abd979415fd8fd5e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } } }
I20260806 22:37:33.912878 3569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:37:33.917107 3627 mvcc.cc:205] Tried to move back new op lower bound from 7315684777449435136 to 7315684776926941184. Current Snapshot: MvccSnapshot[applied={T|T < 7315684777449435136}]
I20260806 22:37:33.922611 3629 mvcc.cc:205] Tried to move back new op lower bound from 7315684777449435136 to 7315684776926941184. Current Snapshot: MvccSnapshot[applied={T|T < 7315684777449435136}]
I20260806 22:37:33.926261 3561 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "291130a0ca4f472abd979415fd8fd5e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } } }
I20260806 22:37:33.926541 3561 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:37:33.932816 3628 mvcc.cc:205] Tried to move back new op lower bound from 7315684777449435136 to 7315684776926941184. Current Snapshot: MvccSnapshot[applied={T|T < 7315684777449435136}]
I20260806 22:37:33.935068 3569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "291130a0ca4f472abd979415fd8fd5e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } } }
I20260806 22:37:33.935317 3569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:37:33.945149 3626 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:37:33.945390 3626 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:37:33.946812 3593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "291130a0ca4f472abd979415fd8fd5e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } } }
I20260806 22:37:33.947244 3593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:33.947759 3593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "291130a0ca4f472abd979415fd8fd5e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291130a0ca4f472abd979415fd8fd5e4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41747 } } peers { permanent_uuid: "17501eba618745a891daef47a8aeef6c" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 33299 } } peers { permanent_uuid: "f3ecbf9a9d9e4efc9e3d58421bfb1a8f" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 39461 } } }
I20260806 22:37:33.948030 3593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:33.949515 3605 catalog_manager.cc:1347] Generated new cluster ID: a972ca3f913342c9a2dae93fc22995a0
I20260806 22:37:33.949631 3605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:33.982671 3605 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:33.985988 3605 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:34.015854 3605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 291130a0ca4f472abd979415fd8fd5e4: Generated new TSK 0
I20260806 22:37:34.017475 3605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:37:34.885576 3620 catalog_manager.cc:1259] Loaded cluster ID: a972ca3f913342c9a2dae93fc22995a0
I20260806 22:37:34.885761 3620 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f: loading cluster ID for follower catalog manager: success
I20260806 22:37:34.889588 3620 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f: acquiring CA information for follower catalog manager: success
I20260806 22:37:34.892139 3620 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f3ecbf9a9d9e4efc9e3d58421bfb1a8f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:37:34.948107 3626 catalog_manager.cc:1259] Loaded cluster ID: a972ca3f913342c9a2dae93fc22995a0
I20260806 22:37:34.948246 3626 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c: loading cluster ID for follower catalog manager: success
I20260806 22:37:34.951224 3626 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c: acquiring CA information for follower catalog manager: success
I20260806 22:37:34.953414 3626 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 17501eba618745a891daef47a8aeef6c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:37:35.192838 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 3210
I20260806 22:37:35.195379 3306 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:35.327430 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 3210
I20260806 22:37:35.361740 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 3322
I20260806 22:37:35.363399 3413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:35.489539 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 3322
I20260806 22:37:35.527048 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 3439
I20260806 22:37:35.528617 3533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:35.654645 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 3439
2026-08-06T22:37:35Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5916 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:37:35 dist-test-slave-h3ft krb5kdc[3653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:37:35 dist-test-slave-h3ft krb5kdc[3653](info): set up 2 sockets
Aug 06 22:37:35 dist-test-slave-h3ft krb5kdc[3653](info): commencing operation
krb5kdc: starting...
W20260806 22:37:37.117298 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.399s user 0.000s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:37:37 dist-test-slave-h3ft krb5kdc[3653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:37:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:37:37.528847 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:38707
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:46187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:38707
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:37.983120 3675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:37.983573 3675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:37.983680 3675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:37.992836 3675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:37.993031 3675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:37.993099 3675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:37.993165 3675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:37.993218 3675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:38.005589 3675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:46187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:38707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:38707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:38.007608 3675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:38.009394 3675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:38.025384 3680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:38.025311 3684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:38.025343 3681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:38.026862 3675 server_base.cc:1034] running on GCE node
I20260806 22:37:38.028045 3675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:38.030100 3675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:38.031373 3675 hybrid_clock.cc:648] HybridClock initialized: now 1786055858031311 us; error 51 us; skew 500 ppm
I20260806 22:37:38.031829 3675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:38 dist-test-slave-h3ft krb5kdc[3653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055858, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:38.042119 3675 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:37:38.044416 3675 webserver.cc:460] Webserver started at http://127.15.198.190:38993/ using document root <none> and password file <none>
I20260806 22:37:38.045396 3675 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:38.045529 3675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:38.045912 3675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:38.050104 3675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "39cba7aa39fe450c8edc04a7bd0815fc"
format_stamp: "Formatted at 2026-08-06 22:37:38 on dist-test-slave-h3ft"
server_key: "eeca435a5be92de0e0982bd784ae3733"
server_key_iv: "fe5681b39981724f13d8327191202495"
server_key_version: "encryptionkey@0"
I20260806 22:37:38.051235 3675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "39cba7aa39fe450c8edc04a7bd0815fc"
format_stamp: "Formatted at 2026-08-06 22:37:38 on dist-test-slave-h3ft"
server_key: "eeca435a5be92de0e0982bd784ae3733"
server_key_iv: "fe5681b39981724f13d8327191202495"
server_key_version: "encryptionkey@0"
I20260806 22:37:38.059183 3675 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 22:37:38.064180 3695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:38.066267 3675 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:37:38.066527 3675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "39cba7aa39fe450c8edc04a7bd0815fc"
format_stamp: "Formatted at 2026-08-06 22:37:38 on dist-test-slave-h3ft"
server_key: "eeca435a5be92de0e0982bd784ae3733"
server_key_iv: "fe5681b39981724f13d8327191202495"
server_key_version: "encryptionkey@0"
I20260806 22:37:38.066784 3675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:38.096134 3675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:38.103340 3675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:38.103792 3675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:38.128531 3675 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:38707
I20260806 22:37:38.128551 3769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:38707 every 8 connection(s)
I20260806 22:37:38.130473 3675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:38.134119 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 3675
I20260806 22:37:38.134507 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:37:38.135429 16154 external_mini_cluster.cc:1428] Setting key c4e0697071c307cacab201fdae841d19
I20260806 22:37:38.136458 3771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 22:37:38 dist-test-slave-h3ft krb5kdc[3653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:37:38.154404 3771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc: Bootstrap starting.
I20260806 22:37:38.158777 3771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:38.160527 3771 log.cc:824] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:38.164451 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:38.137847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38166 (local address 127.15.198.190:38707)
0806 22:37:38.138859 (+ 1012us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:38.138873 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:38.138926 (+ 53us) server_negotiation.cc:407] Connection header received
0806 22:37:38.140841 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:38.140868 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:38.141366 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:38.141815 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:38.144114 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.145748 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.147762 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.148553 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.148699 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:38.154437 (+ 5738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:38.154462 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:38.154484 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:38.154570 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:38.158170 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:38.159148 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:38.159160 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:38.159171 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:38.159515 (+ 344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:38.160022 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:38.160027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:38.160030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:38.160963 (+ 933us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:38.161399 (+ 436us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:38.162053 (+ 654us) server_negotiation.cc:299] Negotiation successful
0806 22:37:38.162649 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":751,"thread_start_us":478,"threads_started":1}
I20260806 22:37:38.165449 3771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc: No bootstrap required, opened a new log
I20260806 22:37:38.169700 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:37:38.176038 3771 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39cba7aa39fe450c8edc04a7bd0815fc" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38707 } }
I20260806 22:37:38.176614 3771 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:38.176767 3771 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39cba7aa39fe450c8edc04a7bd0815fc, State: Initialized, Role: FOLLOWER
I20260806 22:37:38.177606 3771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39cba7aa39fe450c8edc04a7bd0815fc" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38707 } }
I20260806 22:37:38.177944 3771 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:38.178117 3771 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:38.178347 3771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:38.181274 3771 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39cba7aa39fe450c8edc04a7bd0815fc" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38707 } }
I20260806 22:37:38.182086 3771 leader_election.cc:302] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39cba7aa39fe450c8edc04a7bd0815fc; no voters:
I20260806 22:37:38.182971 3771 leader_election.cc:288] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:38.183300 3777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:38.184103 3777 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [term 1 LEADER]: Becoming Leader. State: Replica: 39cba7aa39fe450c8edc04a7bd0815fc, State: Running, Role: LEADER
I20260806 22:37:38.184782 3777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39cba7aa39fe450c8edc04a7bd0815fc" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38707 } }
I20260806 22:37:38.185976 3771 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:38.189138 3779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39cba7aa39fe450c8edc04a7bd0815fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39cba7aa39fe450c8edc04a7bd0815fc" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38707 } } }
I20260806 22:37:38.189845 3779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:38.188975 3782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39cba7aa39fe450c8edc04a7bd0815fc. Latest consensus state: current_term: 1 leader_uuid: "39cba7aa39fe450c8edc04a7bd0815fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39cba7aa39fe450c8edc04a7bd0815fc" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38707 } } }
I20260806 22:37:38.190652 3782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:38.191771 3790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:38.198647 3790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:38.214002 3790 catalog_manager.cc:1347] Generated new cluster ID: a1d5a098567043ceaa063309c3b1a5ed
I20260806 22:37:38.214221 3790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:38.240104 3790 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:38.241927 3790 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:38.253595 3790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39cba7aa39fe450c8edc04a7bd0815fc: Generated new TSK 0
I20260806 22:37:38.254946 3790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:37:38.286072 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 3675
I20260806 22:37:38.287829 3763 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:38.397552 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 3675
2026-08-06T22:37:38Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2737 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:37:38 dist-test-slave-h3ft krb5kdc[3810](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:37:38 dist-test-slave-h3ft krb5kdc[3810](info): set up 2 sockets
Aug 06 22:37:38 dist-test-slave-h3ft krb5kdc[3810](info): commencing operation
krb5kdc: starting...
W20260806 22:37:39.800470 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.346s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:37:40 dist-test-slave-h3ft krb5kdc[3810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:37:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:37:40.211833 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:38391
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:38391
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:40.660683 3828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:40.661160 3828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:40.661279 3828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:40.669632 3828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:40.669826 3828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:40.669879 3828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:40.669936 3828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:40.669992 3828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:40.681845 3828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:38391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:38391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:40.683876 3828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:40.685632 3828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:40.701398 3835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:40.701500 3841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:40.701709 3834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:40.703516 3828 server_base.cc:1034] running on GCE node
I20260806 22:37:40.704768 3828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:40.707053 3828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:40.708446 3828 hybrid_clock.cc:648] HybridClock initialized: now 1786055860708347 us; error 58 us; skew 500 ppm
I20260806 22:37:40.709024 3828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:40 dist-test-slave-h3ft krb5kdc[3810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055860, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:40.714223 3828 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:37:40.716631 3828 webserver.cc:460] Webserver started at http://127.15.198.190:36783/ using document root <none> and password file <none>
I20260806 22:37:40.717710 3828 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:40.717972 3828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:40.718433 3828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:40.722875 3828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "73dc724fc1664869a3e9b0b34b949f54"
format_stamp: "Formatted at 2026-08-06 22:37:40 on dist-test-slave-h3ft"
server_key: "b0747b1c7a162e55e1a256403509b316"
server_key_iv: "06735747abe8e9d65283d6fa655d8d79"
server_key_version: "encryptionkey@0"
I20260806 22:37:40.724102 3828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "73dc724fc1664869a3e9b0b34b949f54"
format_stamp: "Formatted at 2026-08-06 22:37:40 on dist-test-slave-h3ft"
server_key: "b0747b1c7a162e55e1a256403509b316"
server_key_iv: "06735747abe8e9d65283d6fa655d8d79"
server_key_version: "encryptionkey@0"
I20260806 22:37:40.732033 3828 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 22:37:40.737041 3852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:40.739177 3828 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.007s sys 0.000s
I20260806 22:37:40.739488 3828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "73dc724fc1664869a3e9b0b34b949f54"
format_stamp: "Formatted at 2026-08-06 22:37:40 on dist-test-slave-h3ft"
server_key: "b0747b1c7a162e55e1a256403509b316"
server_key_iv: "06735747abe8e9d65283d6fa655d8d79"
server_key_version: "encryptionkey@0"
I20260806 22:37:40.739821 3828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:40.762657 3828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:40.768882 3828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:40.769291 3828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:40.793887 3828 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:38391
I20260806 22:37:40.793938 3924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:38391 every 8 connection(s)
I20260806 22:37:40.795884 3828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:40.798055 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 3828
I20260806 22:37:40.798347 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:37:40.799342 16154 external_mini_cluster.cc:1428] Setting key 9a5e5136503c047fcb887c6a1f23993c
I20260806 22:37:40.802948 3925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:40.824324 3925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54: Bootstrap starting.
Aug 06 22:37:40 dist-test-slave-h3ft krb5kdc[3810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:37:40.829556 3925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:40.831252 3925 log.cc:824] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:40.835636 3925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54: No bootstrap required, opened a new log
I20260806 22:37:40.837957 3930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.809671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40876 (local address 127.15.198.190:38391)
0806 22:37:40.810778 (+ 1107us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:40.810794 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:40.810867 (+ 73us) server_negotiation.cc:407] Connection header received
0806 22:37:40.812980 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:40.813015 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:40.813606 (+ 591us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:40.814087 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:40.816580 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.818171 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.820679 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.821703 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.821901 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.828313 (+ 6412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:40.828325 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:40.828352 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:40.828429 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:40.832588 (+ 4159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:40.833603 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:40.833614 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:40.833623 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:40.833849 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:40.834337 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:40.834342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:40.834344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:40.835111 (+ 767us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:40.835510 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:40.836038 (+ 528us) server_negotiation.cc:299] Negotiation successful
0806 22:37:40.836664 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":808,"thread_start_us":525,"threads_started":1}
I20260806 22:37:40.849207 3925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } }
I20260806 22:37:40.849876 3925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:40.850010 3925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73dc724fc1664869a3e9b0b34b949f54, State: Initialized, Role: FOLLOWER
I20260806 22:37:40.851068 3925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } }
I20260806 22:37:40.851449 3925 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:40.851661 3925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:40.851913 3925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:40.855408 3925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } }
I20260806 22:37:40.856283 3925 leader_election.cc:302] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73dc724fc1664869a3e9b0b34b949f54; no voters:
I20260806 22:37:40.857090 3925 leader_election.cc:288] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:40.857265 3933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:40.858254 3933 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 1 LEADER]: Becoming Leader. State: Replica: 73dc724fc1664869a3e9b0b34b949f54, State: Running, Role: LEADER
I20260806 22:37:40.858958 3933 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } }
I20260806 22:37:40.860083 3925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:40.862917 3934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73dc724fc1664869a3e9b0b34b949f54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } } }
I20260806 22:37:40.863023 3935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73dc724fc1664869a3e9b0b34b949f54. Latest consensus state: current_term: 1 leader_uuid: "73dc724fc1664869a3e9b0b34b949f54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } } }
I20260806 22:37:40.863412 3934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:40.863466 3935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:40.868080 3940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:40.875425 3940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:40.887095 3940 catalog_manager.cc:1347] Generated new cluster ID: 2b2d584aa03a466ba0e2824592d0ddf7
I20260806 22:37:40.887274 3940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:40.909698 3940 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:40.912644 3940 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:40.926312 3940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54: Generated new TSK 0
I20260806 22:37:40.927753 3940 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
I20260806 22:37:41.028120 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:38391
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:41.497184 3969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:41.497591 3969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:41.497768 3969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:41.506351 3969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:41.506546 3969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:41.506750 3969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:37:41.518639 3969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:38391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:41.520710 3969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:41.522643 3969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:41.541069 3975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:41.541174 3977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:41.541121 3979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:41.545713 3969 server_base.cc:1034] running on GCE node
I20260806 22:37:41.546825 3969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:41.548328 3969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:41.549631 3969 hybrid_clock.cc:648] HybridClock initialized: now 1786055861549569 us; error 55 us; skew 500 ppm
I20260806 22:37:41.550122 3969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:41 dist-test-slave-h3ft krb5kdc[3810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055861, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:41.554803 3969 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:37:41.557353 3969 webserver.cc:460] Webserver started at http://127.15.198.129:42073/ using document root <none> and password file <none>
I20260806 22:37:41.558781 3969 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:41.558985 3969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:41.559530 3969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:41.564353 3969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-h3ft"
server_key: "34d906cd82fafd7e617a6cfc84151471"
server_key_iv: "d2ed13d13bd2cbdc13962e3b681347e7"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.565440 3969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-h3ft"
server_key: "34d906cd82fafd7e617a6cfc84151471"
server_key_iv: "d2ed13d13bd2cbdc13962e3b681347e7"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.573559 3969 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.008s sys 0.000s
I20260806 22:37:41.580077 3988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:41.582623 3969 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 22:37:41.582967 3969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-h3ft"
server_key: "34d906cd82fafd7e617a6cfc84151471"
server_key_iv: "d2ed13d13bd2cbdc13962e3b681347e7"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.583371 3969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:41.646559 3969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:41.652817 3969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:41.653252 3969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:41.654663 3969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:41.657198 3969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:41.657341 3969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:41.657531 3969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:41.657611 3969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:41.702193 3969 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:38563
I20260806 22:37:41.702232 4152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:38563 every 8 connection(s)
I20260806 22:37:41.704178 3969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:37:41.705518 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 3969
I20260806 22:37:41.705845 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:37:41.706939 16154 external_mini_cluster.cc:1428] Setting key 1ef32ce7a8d0d7544b5046d6ae3f3e5b
Aug 06 22:37:41 dist-test-slave-h3ft krb5kdc[3810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055861, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:37:41.740477 4155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.709189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:41941 (local address 127.15.198.190:38391)
0806 22:37:41.709884 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:41.709889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:41.711905 (+ 2016us) server_negotiation.cc:407] Connection header received
0806 22:37:41.714725 (+ 2820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:41.714735 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:41.714870 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:41.715050 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:41.717489 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:41.718904 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:41.722106 (+ 3202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:41.723207 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:41.723479 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:41.730074 (+ 6595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:41.730082 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:41.730086 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:41.730242 (+ 156us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:41.734470 (+ 4228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:41.736238 (+ 1768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:41.736245 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:41.736248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:41.736496 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:41.737373 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:41.737378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:41.737380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:41.737797 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:41.737983 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:41.739844 (+ 1861us) server_negotiation.cc:299] Negotiation successful
0806 22:37:41.740141 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":402,"threads_started":1}
I20260806 22:37:41.742218 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.710264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:41941)
0806 22:37:41.711634 (+ 1370us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:41.711717 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:41.714250 (+ 2533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:41.715449 (+ 1199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:41.715461 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:41.716246 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:41.717169 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:41.717198 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:41.719237 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:41.719243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:41.721749 (+ 2506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:41.721776 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:41.723708 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:41.723715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:41.724050 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:41.725265 (+ 1215us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:41.725307 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:41.729539 (+ 4232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:41.734927 (+ 5388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:41.734945 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:41.734973 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:41.735954 (+ 981us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:41.736702 (+ 748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:41.736711 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:41.736717 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:41.737113 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:41.738009 (+ 896us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:41.738023 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:41.738775 (+ 752us) client_negotiation.cc:769] Sending connection context
0806 22:37:41.739301 (+ 526us) client_negotiation.cc:240] Negotiation successful
0806 22:37:41.740162 (+ 861us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":962,"thread_start_us":589,"threads_started":1}
I20260806 22:37:41.745399 4153 heartbeater.cc:347] Connected to a master server at 127.15.198.190:38391
I20260806 22:37:41.746063 4153 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:41.747399 4153 heartbeater.cc:511] Master 127.15.198.190:38391 requested a full tablet report, sending...
I20260806 22:37:41.751492 3875 ts_manager.cc:194] Registered new tserver with Master: 1ac48cc3c2f3475a99fb6a8cc5e3727b (127.15.198.129:38563)
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
I20260806 22:37:41.753830 3875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:41941
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
I20260806 22:37:41.809274 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:38391
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:42.277505 4160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:42.278075 4160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:42.278280 4160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:42.288968 4160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:42.289212 4160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:42.289505 4160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:37:42.304006 4160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:38391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:42.306051 4160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:42.307984 4160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:42.325425 4167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:42.325304 4170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:42.325357 4168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:42.327235 4160 server_base.cc:1034] running on GCE node
I20260806 22:37:42.328197 4160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:42.329638 4160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:42.330922 4160 hybrid_clock.cc:648] HybridClock initialized: now 1786055862330860 us; error 101 us; skew 500 ppm
I20260806 22:37:42.331394 4160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:42 dist-test-slave-h3ft krb5kdc[3810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055862, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:42.335860 4160 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:37:42.338147 4160 webserver.cc:460] Webserver started at http://127.15.198.130:33341/ using document root <none> and password file <none>
I20260806 22:37:42.339243 4160 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:42.339407 4160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:42.339828 4160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:42.344039 4160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "d5bb01607ba841f3bac49253ca74a417"
format_stamp: "Formatted at 2026-08-06 22:37:42 on dist-test-slave-h3ft"
server_key: "b2fa861ed409729b29ae4a5cbde2761d"
server_key_iv: "1030e928d6cc38feba095f9920f08c79"
server_key_version: "encryptionkey@0"
I20260806 22:37:42.345273 4160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "d5bb01607ba841f3bac49253ca74a417"
format_stamp: "Formatted at 2026-08-06 22:37:42 on dist-test-slave-h3ft"
server_key: "b2fa861ed409729b29ae4a5cbde2761d"
server_key_iv: "1030e928d6cc38feba095f9920f08c79"
server_key_version: "encryptionkey@0"
I20260806 22:37:42.352986 4160 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260806 22:37:42.358343 4183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:42.360802 4160 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260806 22:37:42.361071 4160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "d5bb01607ba841f3bac49253ca74a417"
format_stamp: "Formatted at 2026-08-06 22:37:42 on dist-test-slave-h3ft"
server_key: "b2fa861ed409729b29ae4a5cbde2761d"
server_key_iv: "1030e928d6cc38feba095f9920f08c79"
server_key_version: "encryptionkey@0"
I20260806 22:37:42.361387 4160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:42.383975 4160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:42.390533 4160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:42.390900 4160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:42.392308 4160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:42.394219 4160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:42.394310 4160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:42.394423 4160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:42.394477 4160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:42.436213 4160 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:43697
I20260806 22:37:42.436266 4339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:43697 every 8 connection(s)
I20260806 22:37:42.438026 4160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:37:42.447721 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 4160
I20260806 22:37:42.448021 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:37:42.448695 16154 external_mini_cluster.cc:1428] Setting key 98d0ac34fe2358b10384607697c85c37
Aug 06 22:37:42 dist-test-slave-h3ft krb5kdc[3810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055862, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:37:42.469189 4346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:42.441459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:48103 (local address 127.15.198.190:38391)
0806 22:37:42.442037 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:42.442042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:42.443628 (+ 1586us) server_negotiation.cc:407] Connection header received
0806 22:37:42.445999 (+ 2371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:42.446005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:42.446122 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:42.446273 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:42.449020 (+ 2747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.450148 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.452601 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.453109 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.453228 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:42.459166 (+ 5938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:42.459173 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:42.459177 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:42.459252 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:42.463779 (+ 4527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:42.465082 (+ 1303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:42.465092 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:42.465099 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:42.465494 (+ 395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:42.466372 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:42.466381 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:42.466387 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:42.466923 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:42.467187 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:42.468416 (+ 1229us) server_negotiation.cc:299] Negotiation successful
0806 22:37:42.468830 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":278,"threads_started":1}
I20260806 22:37:42.471338 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:42.442202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:48103)
0806 22:37:42.443237 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:42.443302 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:42.445585 (+ 2283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:42.446599 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:42.446610 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:42.447593 (+ 983us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:42.448736 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:42.448774 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.450360 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.450365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:42.452408 (+ 2043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:42.452430 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.453414 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.453425 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:42.453855 (+ 430us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:42.454927 (+ 1072us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:42.455123 (+ 196us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:42.458645 (+ 3522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:42.464115 (+ 5470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:42.464129 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:42.464152 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:42.464869 (+ 717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:42.465695 (+ 826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:42.465699 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:42.465702 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:42.466196 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:42.467139 (+ 943us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:42.467150 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:42.467872 (+ 722us) client_negotiation.cc:769] Sending connection context
0806 22:37:42.468211 (+ 339us) client_negotiation.cc:240] Negotiation successful
0806 22:37:42.469175 (+ 964us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":338,"threads_started":1}
I20260806 22:37:42.474110 4340 heartbeater.cc:347] Connected to a master server at 127.15.198.190:38391
I20260806 22:37:42.474649 4340 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:42.475930 4340 heartbeater.cc:511] Master 127.15.198.190:38391 requested a full tablet report, sending...
I20260806 22:37:42.478384 3875 ts_manager.cc:194] Registered new tserver with Master: d5bb01607ba841f3bac49253ca74a417 (127.15.198.130:43697)
I20260806 22:37:42.479624 3875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:48103
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
I20260806 22:37:42.544966 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:38391
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:37:42.759289 4153 heartbeater.cc:503] Master 127.15.198.190:38391 was elected leader, sending a full tablet report...
W20260806 22:37:42.997390 4351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:42.997869 4351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:42.998050 4351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:43.006613 4351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:43.006860 4351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:43.007140 4351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:37:43.018919 4351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:38391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:43.021013 4351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:43.022838 4351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:43.039088 4357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:43.039180 4358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:43.039132 4362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:43.040447 4351 server_base.cc:1034] running on GCE node
I20260806 22:37:43.041298 4351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:43.042546 4351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:43.043869 4351 hybrid_clock.cc:648] HybridClock initialized: now 1786055863043743 us; error 82 us; skew 500 ppm
I20260806 22:37:43.044324 4351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:43 dist-test-slave-h3ft krb5kdc[3810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055863, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:43.051698 4351 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:37:43.053841 4351 webserver.cc:460] Webserver started at http://127.15.198.131:40295/ using document root <none> and password file <none>
I20260806 22:37:43.054792 4351 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:43.054942 4351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:43.055374 4351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:43.059237 4351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "353f00395f5a4b01b62aa5d8da2b12be"
format_stamp: "Formatted at 2026-08-06 22:37:43 on dist-test-slave-h3ft"
server_key: "7880c83fc596fa97bca030ccadc83278"
server_key_iv: "e39c9835b1ce9df21ecfb9daedaea55d"
server_key_version: "encryptionkey@0"
I20260806 22:37:43.060279 4351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "353f00395f5a4b01b62aa5d8da2b12be"
format_stamp: "Formatted at 2026-08-06 22:37:43 on dist-test-slave-h3ft"
server_key: "7880c83fc596fa97bca030ccadc83278"
server_key_iv: "e39c9835b1ce9df21ecfb9daedaea55d"
server_key_version: "encryptionkey@0"
I20260806 22:37:43.067771 4351 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 22:37:43.072903 4370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:43.075376 4351 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.007s sys 0.000s
I20260806 22:37:43.075651 4351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "353f00395f5a4b01b62aa5d8da2b12be"
format_stamp: "Formatted at 2026-08-06 22:37:43 on dist-test-slave-h3ft"
server_key: "7880c83fc596fa97bca030ccadc83278"
server_key_iv: "e39c9835b1ce9df21ecfb9daedaea55d"
server_key_version: "encryptionkey@0"
I20260806 22:37:43.075928 4351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:43.098212 4351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:43.104673 4351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:43.105075 4351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:43.106554 4351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:43.108556 4351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:43.108650 4351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:43.108836 4351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:43.108897 4351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:43.151290 4351 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:44049
I20260806 22:37:43.151317 4530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:44049 every 8 connection(s)
I20260806 22:37:43.153084 4351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:37:43.158818 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 4351
I20260806 22:37:43.159127 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:37:43.159828 16154 external_mini_cluster.cc:1428] Setting key 52aae215efbcd0bd968a1ae687e21852
Aug 06 22:37:43 dist-test-slave-h3ft krb5kdc[3810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055863, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:37:43.178725 4533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:43.156541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:47337 (local address 127.15.198.190:38391)
0806 22:37:43.157176 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:43.157181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:43.158188 (+ 1007us) server_negotiation.cc:407] Connection header received
0806 22:37:43.161351 (+ 3163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:43.161356 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:43.161475 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:43.161616 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:43.163654 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.164509 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.166477 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.166991 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.167214 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:43.171744 (+ 4530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:43.171750 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:43.171754 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:43.171796 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:43.174903 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:43.175830 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:43.175838 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:43.175847 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:43.175995 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:43.176519 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:43.176523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:43.176525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:43.176868 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:43.177087 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:43.177981 (+ 894us) server_negotiation.cc:299] Negotiation successful
0806 22:37:43.178264 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":322,"threads_started":1}
I20260806 22:37:43.180182 4534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:43.157104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:47337)
0806 22:37:43.157987 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:43.158042 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:43.160929 (+ 2887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:43.161924 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:43.161934 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:43.162606 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:43.163460 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:43.163483 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.164714 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.164719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:43.166286 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:43.166303 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.167366 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.167386 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:43.167728 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:43.168567 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:43.168601 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:43.171444 (+ 2843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:43.175115 (+ 3671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:43.175128 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:43.175154 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:43.175683 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:43.176122 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:43.176127 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:43.176131 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:43.176369 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:43.177033 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:43.177043 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:43.177534 (+ 491us) client_negotiation.cc:769] Sending connection context
0806 22:37:43.177865 (+ 331us) client_negotiation.cc:240] Negotiation successful
0806 22:37:43.178601 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":343,"threads_started":1}
I20260806 22:37:43.182034 4531 heartbeater.cc:347] Connected to a master server at 127.15.198.190:38391
I20260806 22:37:43.182458 4531 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:43.183588 4531 heartbeater.cc:511] Master 127.15.198.190:38391 requested a full tablet report, sending...
I20260806 22:37:43.185627 3875 ts_manager.cc:194] Registered new tserver with Master: 353f00395f5a4b01b62aa5d8da2b12be (127.15.198.131:44049)
I20260806 22:37:43.186650 3875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:47337
I20260806 22:37:43.189652 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:37:43.212939 4533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:43.195754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40880 (local address 127.15.198.190:38391)
0806 22:37:43.196146 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:43.196156 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:43.196489 (+ 333us) server_negotiation.cc:407] Connection header received
0806 22:37:43.196677 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:43.196682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:43.196781 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:43.196881 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:43.198706 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.199731 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.201361 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.201823 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.202001 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:43.205383 (+ 3382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:43.205389 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:43.205393 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:43.205442 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:43.209408 (+ 3966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:43.210446 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:43.210452 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:43.210454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:43.210623 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:43.211105 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:43.211110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:43.211116 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:43.211591 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:43.211831 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:43.212339 (+ 508us) server_negotiation.cc:299] Negotiation successful
0806 22:37:43.212627 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 22:37:43.219214 3875 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:37:43.227942 3875 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:43.264808 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:43.250107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47822 (local address 127.15.198.130:43697)
0806 22:37:43.251667 (+ 1560us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:43.251682 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:43.254503 (+ 2821us) server_negotiation.cc:407] Connection header received
0806 22:37:43.254656 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:43.254667 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:43.254926 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:43.255192 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:43.256333 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.258004 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.261308 (+ 3304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.262568 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.262713 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:43.262900 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:43.264151 (+ 1251us) server_negotiation.cc:299] Negotiation successful
0806 22:37:43.264409 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1289,"thread_start_us":1118,"threads_started":1}
I20260806 22:37:43.264788 4552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:43.248254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:44049 (local address 127.0.0.1:51610)
0806 22:37:43.249556 (+ 1302us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:43.249619 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:43.249840 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:43.254876 (+ 5036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:43.254887 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:43.254926 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:43.255541 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:43.255558 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.258041 (+ 2483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.258047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:43.260946 (+ 2899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:43.260968 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.263312 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.263317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:43.263571 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:43.263733 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 22:37:43.263906 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 22:37:43.264206 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":890,"mutex_wait_us":301,"thread_start_us":5412,"threads_started":1}
I20260806 22:37:43.265269 4554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:43.248939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36046 (local address 127.15.198.129:38563)
0806 22:37:43.252412 (+ 3473us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:43.252425 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:43.253730 (+ 1305us) server_negotiation.cc:407] Connection header received
0806 22:37:43.254175 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:43.254186 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:43.254546 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:43.254802 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:43.256246 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.257476 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.261380 (+ 3904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.263523 (+ 2143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.263797 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:43.263994 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:43.264602 (+ 608us) server_negotiation.cc:299] Negotiation successful
0806 22:37:43.264828 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3230,"thread_start_us":3004,"threads_started":1}
I20260806 22:37:43.265820 4550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:43.249429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:43697 (local address 127.0.0.1:47822)
0806 22:37:43.254096 (+ 4667us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:43.254114 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:43.254281 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:43.255432 (+ 1151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:43.255436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:43.255461 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:43.256131 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:43.256144 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.258186 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.258191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:43.261095 (+ 2904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:43.261106 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.263185 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.263195 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:43.263564 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:43.263733 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 22:37:43.263906 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 22:37:43.265332 (+ 1426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4524,"mutex_wait_us":1527,"thread_start_us":3784,"threads_started":1}
I20260806 22:37:43.266250 4555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:43.248268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:38563 (local address 127.0.0.1:36046)
0806 22:37:43.253556 (+ 5288us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:43.253586 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:43.253890 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:43.255212 (+ 1322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:43.255217 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:43.255239 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:43.255958 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:43.255989 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.257651 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.257655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:43.261051 (+ 3396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:43.261070 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.263797 (+ 2727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.263805 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:43.264358 (+ 553us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:43.264396 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:37:43.265508 (+ 1112us) client_negotiation.cc:240] Negotiation successful
0806 22:37:43.265676 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5128,"mutex_wait_us":416,"thread_start_us":354,"threads_started":1}
I20260806 22:37:43.266847 4553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:43.248468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51610 (local address 127.15.198.131:44049)
0806 22:37:43.253174 (+ 4706us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:43.253186 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:43.253241 (+ 55us) server_negotiation.cc:407] Connection header received
0806 22:37:43.253409 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:43.253420 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:43.254568 (+ 1148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:43.255622 (+ 1054us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:43.256006 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.257854 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.261195 (+ 3341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.262989 (+ 1794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.263312 (+ 323us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:43.263515 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:43.266161 (+ 2646us) server_negotiation.cc:299] Negotiation successful
0806 22:37:43.266389 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4484,"thread_start_us":4256,"threads_started":1}
I20260806 22:37:43.270833 4437 tablet_service.cc:1467] Processing CreateTablet for tablet 717b8171487e43a4b6a59a41450a106c (DEFAULT_TABLE table=test-table [id=0418fc34afec4b09ad5f20ced2b8ca8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:43.270833 4250 tablet_service.cc:1467] Processing CreateTablet for tablet 717b8171487e43a4b6a59a41450a106c (DEFAULT_TABLE table=test-table [id=0418fc34afec4b09ad5f20ced2b8ca8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:43.270848 4056 tablet_service.cc:1467] Processing CreateTablet for tablet 717b8171487e43a4b6a59a41450a106c (DEFAULT_TABLE table=test-table [id=0418fc34afec4b09ad5f20ced2b8ca8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:43.273005 4250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 717b8171487e43a4b6a59a41450a106c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:43.273066 4437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 717b8171487e43a4b6a59a41450a106c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:43.273459 4056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 717b8171487e43a4b6a59a41450a106c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:43.291134 4557 tablet_bootstrap.cc:492] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b: Bootstrap starting.
I20260806 22:37:43.292963 4559 tablet_bootstrap.cc:492] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be: Bootstrap starting.
I20260806 22:37:43.293843 4558 tablet_bootstrap.cc:492] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417: Bootstrap starting.
I20260806 22:37:43.296388 4557 tablet_bootstrap.cc:654] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:43.298166 4557 log.cc:824] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:43.298352 4559 tablet_bootstrap.cc:654] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:43.298429 4558 tablet_bootstrap.cc:654] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:43.299821 4558 log.cc:824] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:43.299880 4559 log.cc:824] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:43.302821 4557 tablet_bootstrap.cc:492] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b: No bootstrap required, opened a new log
I20260806 22:37:43.303303 4557 ts_tablet_manager.cc:1397] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260806 22:37:43.303764 4559 tablet_bootstrap.cc:492] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be: No bootstrap required, opened a new log
I20260806 22:37:43.304124 4559 ts_tablet_manager.cc:1397] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 22:37:43.305534 4558 tablet_bootstrap.cc:492] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417: No bootstrap required, opened a new log
I20260806 22:37:43.305876 4558 ts_tablet_manager.cc:1397] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.004s
I20260806 22:37:43.314209 4558 raft_consensus.cc:348] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } }
I20260806 22:37:43.314915 4558 raft_consensus.cc:374] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:43.315069 4558 raft_consensus.cc:729] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5bb01607ba841f3bac49253ca74a417, State: Initialized, Role: FOLLOWER
I20260806 22:37:43.316246 4558 consensus_queue.cc:260] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } }
I20260806 22:37:43.316514 4557 raft_consensus.cc:348] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } }
I20260806 22:37:43.316510 4559 raft_consensus.cc:348] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } }
I20260806 22:37:43.317093 4559 raft_consensus.cc:374] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:43.317085 4557 raft_consensus.cc:374] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:43.317209 4557 raft_consensus.cc:729] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ac48cc3c2f3475a99fb6a8cc5e3727b, State: Initialized, Role: FOLLOWER
I20260806 22:37:43.317209 4559 raft_consensus.cc:729] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353f00395f5a4b01b62aa5d8da2b12be, State: Initialized, Role: FOLLOWER
I20260806 22:37:43.318133 4559 consensus_queue.cc:260] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } }
I20260806 22:37:43.318135 4557 consensus_queue.cc:260] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } }
I20260806 22:37:43.319101 4340 heartbeater.cc:503] Master 127.15.198.190:38391 was elected leader, sending a full tablet report...
I20260806 22:37:43.319995 4558 ts_tablet_manager.cc:1428] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417: Time spent starting tablet: real 0.014s user 0.006s sys 0.009s
I20260806 22:37:43.321851 4531 heartbeater.cc:503] Master 127.15.198.190:38391 was elected leader, sending a full tablet report...
I20260806 22:37:43.323395 4557 ts_tablet_manager.cc:1428] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b: Time spent starting tablet: real 0.020s user 0.012s sys 0.008s
I20260806 22:37:43.325302 4559 ts_tablet_manager.cc:1428] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be: Time spent starting tablet: real 0.021s user 0.014s sys 0.004s
I20260806 22:37:43.337370 4567 raft_consensus.cc:482] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:43.337811 4567 raft_consensus.cc:504] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } }
I20260806 22:37:43.340624 4567 leader_election.cc:288] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ac48cc3c2f3475a99fb6a8cc5e3727b (127.15.198.129:38563), d5bb01607ba841f3bac49253ca74a417 (127.15.198.130:43697)
I20260806 22:37:43.348708 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:43.341474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:53843 (local address 127.15.198.130:43697)
0806 22:37:43.341736 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:43.341745 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:43.341783 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:37:43.341919 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:43.341928 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:43.342054 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:43.342202 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:43.342697 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.343983 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.346262 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.347631 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.347813 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:43.347885 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:43.348290 (+ 405us) server_negotiation.cc:299] Negotiation successful
0806 22:37:43.348430 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 22:37:43.348762 4534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:43.341061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:43697 (local address 127.15.198.131:53843)
0806 22:37:43.341480 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:43.341514 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:43.341717 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:43.342250 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:43.342254 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:43.342281 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:43.342483 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:43.342491 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.344149 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.344155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:43.346034 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:43.346054 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.347851 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.347857 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:43.348080 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:43.348123 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:37:43.348234 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 22:37:43.348415 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217}
I20260806 22:37:43.350235 4554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:43.341336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:45973 (local address 127.15.198.129:38563)
0806 22:37:43.341610 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:43.341617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:43.342569 (+ 952us) server_negotiation.cc:407] Connection header received
0806 22:37:43.342830 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:43.342835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:43.342904 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:43.343139 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:43.343877 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.345166 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.347832 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.349195 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.349407 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:43.349499 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:43.349784 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 22:37:43.349943 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 22:37:43.350273 4568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:43.341125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:38563 (local address 127.15.198.131:45973)
0806 22:37:43.342438 (+ 1313us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:43.342456 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:43.342636 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:43.343181 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:43.343186 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:43.343219 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:43.343671 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:43.343684 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.345375 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.345381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:43.347604 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:43.347621 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.349437 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.349443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:43.349610 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:43.349636 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 22:37:43.349752 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:37:43.349899 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1122,"thread_start_us":822,"threads_started":1}
I20260806 22:37:43.350112 4277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "717b8171487e43a4b6a59a41450a106c" candidate_uuid: "353f00395f5a4b01b62aa5d8da2b12be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bb01607ba841f3bac49253ca74a417" is_pre_election: true
I20260806 22:37:43.350899 4277 raft_consensus.cc:2454] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 353f00395f5a4b01b62aa5d8da2b12be in term 0.
I20260806 22:37:43.351675 4088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "717b8171487e43a4b6a59a41450a106c" candidate_uuid: "353f00395f5a4b01b62aa5d8da2b12be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" is_pre_election: true
I20260806 22:37:43.351850 4374 leader_election.cc:302] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353f00395f5a4b01b62aa5d8da2b12be, d5bb01607ba841f3bac49253ca74a417; no voters:
I20260806 22:37:43.352339 4567 raft_consensus.cc:2781] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:43.352351 4088 raft_consensus.cc:2454] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 353f00395f5a4b01b62aa5d8da2b12be in term 0.
I20260806 22:37:43.352524 4567 raft_consensus.cc:482] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:43.352686 4567 raft_consensus.cc:3037] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:43.355942 4567 raft_consensus.cc:504] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } }
I20260806 22:37:43.356746 4567 leader_election.cc:288] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [CANDIDATE]: Term 1 election: Requested vote from peers 1ac48cc3c2f3475a99fb6a8cc5e3727b (127.15.198.129:38563), d5bb01607ba841f3bac49253ca74a417 (127.15.198.130:43697)
I20260806 22:37:43.357373 4088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "717b8171487e43a4b6a59a41450a106c" candidate_uuid: "353f00395f5a4b01b62aa5d8da2b12be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b"
I20260806 22:37:43.357524 4277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "717b8171487e43a4b6a59a41450a106c" candidate_uuid: "353f00395f5a4b01b62aa5d8da2b12be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bb01607ba841f3bac49253ca74a417"
I20260806 22:37:43.357677 4088 raft_consensus.cc:3037] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:43.357805 4277 raft_consensus.cc:3037] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:43.361740 4277 raft_consensus.cc:2454] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 353f00395f5a4b01b62aa5d8da2b12be in term 1.
I20260806 22:37:43.361740 4088 raft_consensus.cc:2454] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 353f00395f5a4b01b62aa5d8da2b12be in term 1.
I20260806 22:37:43.362429 4374 leader_election.cc:302] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353f00395f5a4b01b62aa5d8da2b12be, d5bb01607ba841f3bac49253ca74a417; no voters:
I20260806 22:37:43.362902 4567 raft_consensus.cc:2781] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:43.363615 4567 raft_consensus.cc:686] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [term 1 LEADER]: Becoming Leader. State: Replica: 353f00395f5a4b01b62aa5d8da2b12be, State: Running, Role: LEADER
I20260806 22:37:43.364257 4567 consensus_queue.cc:237] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } }
I20260806 22:37:43.371407 3875 catalog_manager.cc:5495] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be reported cstate change: term changed from 0 to 1, leader changed from <none> to 353f00395f5a4b01b62aa5d8da2b12be (127.15.198.131). New cstate: current_term: 1 leader_uuid: "353f00395f5a4b01b62aa5d8da2b12be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } health_report { overall_health: UNKNOWN } } }
I20260806 22:37:43.396095 4553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:43.389005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51626 (local address 127.15.198.131:44049)
0806 22:37:43.389228 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:43.389233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:43.389264 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:37:43.389542 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:43.389547 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:43.389654 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:43.389826 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:43.390397 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.391350 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.393254 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.393726 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.393853 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:43.394595 (+ 742us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:43.395440 (+ 845us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:43.395584 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:43.395729 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 22:37:43.395829 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 22:37:43.405120 4532 tablet.cc:2369] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:37:43.407744 4277 raft_consensus.cc:1261] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417 [term 1 FOLLOWER]: Refusing update from remote peer 353f00395f5a4b01b62aa5d8da2b12be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:43.407737 4088 raft_consensus.cc:1261] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 1 FOLLOWER]: Refusing update from remote peer 353f00395f5a4b01b62aa5d8da2b12be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:43.409124 4569 consensus_queue.cc:1035] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:43.409606 4567 consensus_queue.cc:1035] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:43.448635 3874 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:43.449061 3874 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:43.454257 3874 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54: Sending DeleteTablet for 3 replicas of tablet 717b8171487e43a4b6a59a41450a106c
I20260806 22:37:43.455942 4437 tablet_service.cc:1514] Processing DeleteTablet for tablet 717b8171487e43a4b6a59a41450a106c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:43 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:51610
I20260806 22:37:43.456971 4056 tablet_service.cc:1514] Processing DeleteTablet for tablet 717b8171487e43a4b6a59a41450a106c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:43 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:36046
I20260806 22:37:43.457414 4583 tablet_replica.cc:330] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be: stopping tablet replica
I20260806 22:37:43.458083 4583 raft_consensus.cc:2229] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:37:43.458959 4250 tablet_service.cc:1514] Processing DeleteTablet for tablet 717b8171487e43a4b6a59a41450a106c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:43 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:47822
I20260806 22:37:43.459151 4583 raft_consensus.cc:2258] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:43.462312 4583 ts_tablet_manager.cc:1905] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:43.466591 4585 tablet_replica.cc:330] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417: stopping tablet replica
I20260806 22:37:43.467432 4585 raft_consensus.cc:2229] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:43.467767 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 3969
I20260806 22:37:43.468518 4585 raft_consensus.cc:2258] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:43.471521 4585 ts_tablet_manager.cc:1905] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:43.474645 4583 ts_tablet_manager.cc:1918] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:43.475100 4583 log.cc:1196] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/717b8171487e43a4b6a59a41450a106c
I20260806 22:37:43.476032 4583 ts_tablet_manager.cc:1939] T 717b8171487e43a4b6a59a41450a106c P 353f00395f5a4b01b62aa5d8da2b12be: Deleting consensus metadata
I20260806 22:37:43.479199 3855 catalog_manager.cc:4841] TS 353f00395f5a4b01b62aa5d8da2b12be (127.15.198.131:44049): tablet 717b8171487e43a4b6a59a41450a106c (table test-table [id=0418fc34afec4b09ad5f20ced2b8ca8a]) successfully deleted
I20260806 22:37:43.480605 4585 ts_tablet_manager.cc:1918] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:43.480842 4585 log.cc:1196] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/717b8171487e43a4b6a59a41450a106c
I20260806 22:37:43.481575 4585 ts_tablet_manager.cc:1939] T 717b8171487e43a4b6a59a41450a106c P d5bb01607ba841f3bac49253ca74a417: Deleting consensus metadata
I20260806 22:37:43.483099 4584 tablet_replica.cc:330] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b: stopping tablet replica
I20260806 22:37:43.483850 4584 raft_consensus.cc:2229] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:43.484287 3856 catalog_manager.cc:4841] TS d5bb01607ba841f3bac49253ca74a417 (127.15.198.130:43697): tablet 717b8171487e43a4b6a59a41450a106c (table test-table [id=0418fc34afec4b09ad5f20ced2b8ca8a]) successfully deleted
I20260806 22:37:43.484757 4584 raft_consensus.cc:2258] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:43.487519 4584 ts_tablet_manager.cc:1905] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 22:37:43 dist-test-slave-h3ft krb5kdc[3810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:37:43.497816 4584 ts_tablet_manager.cc:1918] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:43.498133 4584 log.cc:1196] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/717b8171487e43a4b6a59a41450a106c
I20260806 22:37:43.498829 4584 ts_tablet_manager.cc:1939] T 717b8171487e43a4b6a59a41450a106c P 1ac48cc3c2f3475a99fb6a8cc5e3727b: Deleting consensus metadata
I20260806 22:37:43.501713 3854 catalog_manager.cc:4841] TS 1ac48cc3c2f3475a99fb6a8cc5e3727b (127.15.198.129:38563): tablet 717b8171487e43a4b6a59a41450a106c (table test-table [id=0418fc34afec4b09ad5f20ced2b8ca8a]) successfully deleted
I20260806 22:37:43.504081 4554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:43.469543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36060 (local address 127.15.198.129:38563)
0806 22:37:43.469791 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:43.469799 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:43.482110 (+ 12311us) server_negotiation.cc:407] Connection header received
0806 22:37:43.482487 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:43.482497 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:43.482641 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:43.482770 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:43.485381 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.486466 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.488335 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.488907 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.489067 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:43.494851 (+ 5784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:43.494866 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:43.494887 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:43.494943 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:43.499198 (+ 4255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:43.500230 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:43.500239 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:43.500242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:43.500444 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:43.500933 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:43.500943 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:43.500946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:43.501539 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:43.501944 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:43.503311 (+ 1367us) server_negotiation.cc:299] Negotiation successful
0806 22:37:43.503597 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 22:37:43.505015 4148 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:43.608531 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 3969
I20260806 22:37:43.651580 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 4160
Aug 06 22:37:43 dist-test-slave-h3ft krb5kdc[3810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.130@KRBTEST.COM
I20260806 22:37:43.674669 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:43.652821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47834 (local address 127.15.198.130:43697)
0806 22:37:43.653132 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:43.653139 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:43.653167 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:37:43.653331 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:43.653338 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:43.653453 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:43.653579 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:43.655340 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.656289 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.658610 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.659206 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.659333 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:43.666226 (+ 6893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:43.666252 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:43.666287 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:43.666357 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:43.670723 (+ 4366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:43.671864 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:43.671875 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:43.671878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:43.672049 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:43.672555 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:43.672560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:43.672576 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:43.673102 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:43.673526 (+ 424us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:43.673912 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 22:37:43.674227 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 22:37:43.675714 4335 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:43.780625 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 4160
I20260806 22:37:43.820255 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 4351
Aug 06 22:37:43 dist-test-slave-h3ft krb5kdc[3810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.131@KRBTEST.COM
I20260806 22:37:43.843737 4553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:43.821370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51632 (local address 127.15.198.131:44049)
0806 22:37:43.821622 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:43.821630 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:43.821819 (+ 189us) server_negotiation.cc:407] Connection header received
0806 22:37:43.822120 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:43.822126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:43.822217 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:43.822381 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:43.824299 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.825320 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.827292 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:43.827879 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:43.828073 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:43.834191 (+ 6118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:43.834227 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:43.834261 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:43.834356 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:43.839469 (+ 5113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:43.840624 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:43.840639 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:43.840642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:43.840897 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:43.841379 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:43.841386 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:43.841391 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:43.842128 (+ 737us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:43.842544 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:43.842888 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 22:37:43.843231 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 22:37:43.844848 4523 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:43.994527 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 4351
I20260806 22:37:44.028057 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 3828
I20260806 22:37:44.029610 3919 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:44.164150 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 3828
I20260806 22:37:44.205112 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:38391
--webserver_interface=127.15.198.190
--webserver_port=36783
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:38391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:44.662268 4596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:44.662782 4596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:44.662988 4596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:44.671747 4596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:44.671974 4596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:44.672102 4596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:44.672221 4596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:44.672324 4596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:44.685742 4596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:38391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:38391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=36783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:44.688279 4596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:44.690239 4596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:44.729264 4607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:44.729228 4611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:44.729314 4604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:44.729988 4596 server_base.cc:1034] running on GCE node
I20260806 22:37:44.731565 4596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:44.734022 4596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:44.735500 4596 hybrid_clock.cc:648] HybridClock initialized: now 1786055864735456 us; error 118 us; skew 500 ppm
I20260806 22:37:44.736179 4596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:44 dist-test-slave-h3ft krb5kdc[3810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:44.741623 4596 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:37:44.744189 4596 webserver.cc:460] Webserver started at http://127.15.198.190:36783/ using document root <none> and password file <none>
I20260806 22:37:44.745301 4596 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:44.745520 4596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:44.753911 4596 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:37:44.758270 4619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:44.760540 4596 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:37:44.760847 4596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "73dc724fc1664869a3e9b0b34b949f54"
format_stamp: "Formatted at 2026-08-06 22:37:40 on dist-test-slave-h3ft"
server_key: "b0747b1c7a162e55e1a256403509b316"
server_key_iv: "06735747abe8e9d65283d6fa655d8d79"
server_key_version: "encryptionkey@0"
I20260806 22:37:44.762331 4596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:44.781857 4596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:44.788221 4596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:44.788625 4596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:44.814278 4596 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:38391
I20260806 22:37:44.814298 4686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:38391 every 8 connection(s)
I20260806 22:37:44.816712 4596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:44.819736 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 4596
I20260806 22:37:44.820981 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:38563
--local_ip_for_outbound_sockets=127.15.198.129
--tserver_master_addrs=127.15.198.190:38391
--webserver_port=42073
--webserver_interface=127.15.198.129
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:37:44.830504 4687 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:37:44.837478 4687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54: Bootstrap starting.
I20260806 22:37:44.871901 4687 log.cc:824] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:44.888368 4687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:37:44.889132 4687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54: Bootstrap complete.
I20260806 22:37:44.901358 4687 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } }
I20260806 22:37:44.902601 4687 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73dc724fc1664869a3e9b0b34b949f54, State: Initialized, Role: FOLLOWER
I20260806 22:37:44.903653 4687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } }
I20260806 22:37:44.904013 4687 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:44.904191 4687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:44.904420 4687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:37:44.910760 4687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } }
I20260806 22:37:44.911674 4687 leader_election.cc:302] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73dc724fc1664869a3e9b0b34b949f54; no voters:
I20260806 22:37:44.912604 4687 leader_election.cc:288] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:37:44.912814 4692 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:37:44.916101 4687 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:44.918628 4692 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 2 LEADER]: Becoming Leader. State: Replica: 73dc724fc1664869a3e9b0b34b949f54, State: Running, Role: LEADER
I20260806 22:37:44.919719 4692 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } }
I20260806 22:37:44.923892 4693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "73dc724fc1664869a3e9b0b34b949f54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } } }
I20260806 22:37:44.924424 4693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:44.925215 4694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73dc724fc1664869a3e9b0b34b949f54. Latest consensus state: current_term: 2 leader_uuid: "73dc724fc1664869a3e9b0b34b949f54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } } }
I20260806 22:37:44.925558 4694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:44.938829 4704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:44.951040 4704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:44.953063 4704 catalog_manager.cc:1259] Loaded cluster ID: 2b2d584aa03a466ba0e2824592d0ddf7
I20260806 22:37:44.953228 4704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:44.957695 4704 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 22:37:44.976292 4704 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 22:37:44.989180 4704 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:44.991868 4704 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54: Loaded TSK: 0
I20260806 22:37:44.994071 4704 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:37:45.324869 4689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:45.325331 4689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:45.325548 4689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:45.334367 4689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:45.334554 4689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:45.334759 4689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:37:45.347173 4689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:38563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=42073
--tserver_master_addrs=127.15.198.190:38391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:45.349169 4689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:45.351167 4689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:45.369511 4720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:45.369489 4721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:45.371626 4723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:45.373804 4689 server_base.cc:1034] running on GCE node
I20260806 22:37:45.374904 4689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:45.376585 4689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:45.377926 4689 hybrid_clock.cc:648] HybridClock initialized: now 1786055865377817 us; error 86 us; skew 500 ppm
I20260806 22:37:45.378387 4689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:45 dist-test-slave-h3ft krb5kdc[3810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055865, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:45.383652 4689 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:37:45.385953 4689 webserver.cc:460] Webserver started at http://127.15.198.129:42073/ using document root <none> and password file <none>
I20260806 22:37:45.387146 4689 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:45.387373 4689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:45.395363 4689 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 22:37:45.399559 4735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:45.401787 4689 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 22:37:45.402082 4689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-h3ft"
server_key: "34d906cd82fafd7e617a6cfc84151471"
server_key_iv: "d2ed13d13bd2cbdc13962e3b681347e7"
server_key_version: "encryptionkey@0"
I20260806 22:37:45.403540 4689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:45.417948 4689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:45.424504 4689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:45.424917 4689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:45.426505 4689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:45.428604 4689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:45.428791 4689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:45.428973 4689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:45.429081 4689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:45.472522 4689 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:38563
I20260806 22:37:45.472669 4881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:38563 every 8 connection(s)
I20260806 22:37:45.474852 4689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:37:45.475538 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 4689
I20260806 22:37:45.476531 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:43697
--local_ip_for_outbound_sockets=127.15.198.130
--tserver_master_addrs=127.15.198.190:38391
--webserver_port=33341
--webserver_interface=127.15.198.130
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:37:45 dist-test-slave-h3ft krb5kdc[3810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055865, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:37:45.510592 4885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:45.479395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:41583 (local address 127.15.198.190:38391)
0806 22:37:45.480230 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:45.480252 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:45.481360 (+ 1108us) server_negotiation.cc:407] Connection header received
0806 22:37:45.485479 (+ 4119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:45.485499 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:45.486185 (+ 686us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:45.486836 (+ 651us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:45.489167 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:45.490693 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:45.493450 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:45.494301 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:45.494609 (+ 308us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:45.499873 (+ 5264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:45.499888 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:45.499919 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:45.499980 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:45.503919 (+ 3939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:45.505131 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:45.505142 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:45.505149 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:45.505409 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:45.506002 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:45.506007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:45.506009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:45.506822 (+ 813us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:45.507152 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:45.508194 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0806 22:37:45.508801 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":341,"threads_started":1}
I20260806 22:37:45.510632 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:45.479903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:41583)
0806 22:37:45.481065 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:45.481131 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:45.484441 (+ 3310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:45.487067 (+ 2626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:45.487087 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:45.488033 (+ 946us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:45.488825 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:45.488853 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:45.490965 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:45.490971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:45.493154 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:45.493181 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:45.494660 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:45.494669 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:45.495051 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:45.496111 (+ 1060us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:45.496147 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:45.499433 (+ 3286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:45.504202 (+ 4769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:45.504219 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:45.504237 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:45.504895 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:45.505571 (+ 676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:45.505577 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:45.505581 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:45.505847 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:45.507056 (+ 1209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:45.507067 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:45.507553 (+ 486us) client_negotiation.cc:769] Sending connection context
0806 22:37:45.507914 (+ 361us) client_negotiation.cc:240] Negotiation successful
0806 22:37:45.508801 (+ 887us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":378,"threads_started":1}
I20260806 22:37:45.514838 4883 heartbeater.cc:347] Connected to a master server at 127.15.198.190:38391
I20260806 22:37:45.515436 4883 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:45.516940 4883 heartbeater.cc:511] Master 127.15.198.190:38391 requested a full tablet report, sending...
I20260806 22:37:45.520486 4645 ts_manager.cc:194] Registered new tserver with Master: 1ac48cc3c2f3475a99fb6a8cc5e3727b (127.15.198.129:38563)
I20260806 22:37:45.522404 4645 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:41583
W20260806 22:37:45.952709 4887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:45.953207 4887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:45.953341 4887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:45.961918 4887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:45.962085 4887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:45.962224 4887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:37:45.974617 4887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:43697
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=33341
--tserver_master_addrs=127.15.198.190:38391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:45.976593 4887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:45.978348 4887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:45.996006 4894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:45.996083 4897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:45.996003 4895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:45.998231 4887 server_base.cc:1034] running on GCE node
I20260806 22:37:45.999208 4887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:46.000631 4887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:46.001909 4887 hybrid_clock.cc:648] HybridClock initialized: now 1786055866001827 us; error 66 us; skew 500 ppm
I20260806 22:37:46.002318 4887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:46 dist-test-slave-h3ft krb5kdc[3810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055866, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:46.006814 4887 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:37:46.009028 4887 webserver.cc:460] Webserver started at http://127.15.198.130:33341/ using document root <none> and password file <none>
I20260806 22:37:46.010130 4887 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:46.010324 4887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:46.018185 4887 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 22:37:46.022116 4906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:46.024144 4887 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:37:46.024391 4887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "d5bb01607ba841f3bac49253ca74a417"
format_stamp: "Formatted at 2026-08-06 22:37:42 on dist-test-slave-h3ft"
server_key: "b2fa861ed409729b29ae4a5cbde2761d"
server_key_iv: "1030e928d6cc38feba095f9920f08c79"
server_key_version: "encryptionkey@0"
I20260806 22:37:46.025764 4887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:46.059623 4887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:46.065493 4887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:46.065912 4887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:46.067387 4887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:46.069294 4887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:46.069398 4887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:46.069550 4887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:46.069639 4887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:46.114807 4887 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:43697
I20260806 22:37:46.114873 5066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:43697 every 8 connection(s)
I20260806 22:37:46.116935 4887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:37:46.125994 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 4887
I20260806 22:37:46.126799 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:44049
--local_ip_for_outbound_sockets=127.15.198.131
--tserver_master_addrs=127.15.198.190:38391
--webserver_port=40295
--webserver_interface=127.15.198.131
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:37:46 dist-test-slave-h3ft krb5kdc[3810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055866, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:37:46.144387 5070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:46.120725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:57195 (local address 127.15.198.190:38391)
0806 22:37:46.121402 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:46.121408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:46.122530 (+ 1122us) server_negotiation.cc:407] Connection header received
0806 22:37:46.125544 (+ 3014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:46.125551 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:46.125680 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:46.125828 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:46.128146 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.128960 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.131367 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.131892 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.132064 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:46.137027 (+ 4963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:46.137034 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:46.137037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:46.137088 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:46.140358 (+ 3270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:46.141382 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:46.141388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:46.141391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:46.141561 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:46.142115 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:46.142118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:46.142120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:46.142456 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:46.142636 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:46.143607 (+ 971us) server_negotiation.cc:299] Negotiation successful
0806 22:37:46.143902 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":383,"threads_started":1}
I20260806 22:37:46.145777 5071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:46.121372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:57195)
0806 22:37:46.122289 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:46.122342 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:46.125122 (+ 2780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:46.126210 (+ 1088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:46.126226 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:46.127045 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:46.127888 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:46.127912 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.129164 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.129170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:46.131150 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:46.131168 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.132221 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.132229 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:46.132539 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:46.133576 (+ 1037us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:46.133608 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:46.136619 (+ 3011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:46.140575 (+ 3956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:46.140588 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:46.140620 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:46.141203 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:46.141695 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:46.141701 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:46.141704 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:46.141965 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:46.142605 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:46.142616 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:46.143161 (+ 545us) client_negotiation.cc:769] Sending connection context
0806 22:37:46.143489 (+ 328us) client_negotiation.cc:240] Negotiation successful
0806 22:37:46.144244 (+ 755us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":335,"threads_started":1}
I20260806 22:37:46.148062 5068 heartbeater.cc:347] Connected to a master server at 127.15.198.190:38391
I20260806 22:37:46.148502 5068 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:46.149571 5068 heartbeater.cc:511] Master 127.15.198.190:38391 requested a full tablet report, sending...
I20260806 22:37:46.151788 4645 ts_manager.cc:194] Registered new tserver with Master: d5bb01607ba841f3bac49253ca74a417 (127.15.198.130:43697)
I20260806 22:37:46.152814 4645 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:57195
I20260806 22:37:46.527840 4883 heartbeater.cc:503] Master 127.15.198.190:38391 was elected leader, sending a full tablet report...
W20260806 22:37:46.587278 5072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:46.587774 5072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:46.587980 5072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:46.596704 5072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:46.596920 5072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:46.597134 5072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:37:46.609628 5072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:44049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=40295
--tserver_master_addrs=127.15.198.190:38391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:46.611698 5072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:46.613631 5072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:46.631371 5084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:46.631359 5080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:46.631477 5078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:46.637603 5072 server_base.cc:1034] running on GCE node
I20260806 22:37:46.638542 5072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:46.639927 5072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:46.641157 5072 hybrid_clock.cc:648] HybridClock initialized: now 1786055866641098 us; error 57 us; skew 500 ppm
I20260806 22:37:46.641577 5072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:46 dist-test-slave-h3ft krb5kdc[3810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055866, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:46.645689 5072 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:37:46.647884 5072 webserver.cc:460] Webserver started at http://127.15.198.131:40295/ using document root <none> and password file <none>
I20260806 22:37:46.648913 5072 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:46.649102 5072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:46.656635 5072 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 22:37:46.660545 5093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:46.662346 5072 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 22:37:46.662606 5072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "353f00395f5a4b01b62aa5d8da2b12be"
format_stamp: "Formatted at 2026-08-06 22:37:43 on dist-test-slave-h3ft"
server_key: "7880c83fc596fa97bca030ccadc83278"
server_key_iv: "e39c9835b1ce9df21ecfb9daedaea55d"
server_key_version: "encryptionkey@0"
I20260806 22:37:46.664139 5072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:46.686800 5072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:46.693907 5072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:46.694370 5072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:46.695868 5072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:46.697732 5072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:46.697885 5072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:46.698058 5072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:46.698160 5072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:46.739890 5072 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:44049
I20260806 22:37:46.739952 5256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:44049 every 8 connection(s)
I20260806 22:37:46.742022 5072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:37:46.752467 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 5072
Aug 06 22:37:46 dist-test-slave-h3ft krb5kdc[3810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055866, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:37:46.771649 5260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:46.745773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:52403 (local address 127.15.198.190:38391)
0806 22:37:46.746382 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:46.746388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:46.747807 (+ 1419us) server_negotiation.cc:407] Connection header received
0806 22:37:46.750146 (+ 2339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:46.750154 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:46.750306 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:46.750462 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:46.752760 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.754006 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.756565 (+ 2559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.757305 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.757485 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:46.762521 (+ 5036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:46.762527 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:46.762530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:46.762570 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:46.766694 (+ 4124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:46.768109 (+ 1415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:46.768117 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:46.768121 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:46.768356 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:46.769029 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:46.769037 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:46.769040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:46.769510 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:46.769719 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:46.770792 (+ 1073us) server_negotiation.cc:299] Negotiation successful
0806 22:37:46.771134 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":469,"thread_start_us":294,"threads_started":1}
I20260806 22:37:46.772975 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:46.753808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40888 (local address 127.15.198.190:38391)
0806 22:37:46.754863 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:46.754869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:46.754890 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:37:46.754954 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:46.754959 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:46.755121 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:46.755249 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:46.757479 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.758757 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.760401 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.760884 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.761053 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:46.764746 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:46.764753 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:46.764758 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:46.764837 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:46.769084 (+ 4247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:46.770198 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:46.770206 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:46.770210 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:46.770425 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:46.770925 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:46.770931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:46.770935 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:46.771310 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:46.771616 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:46.772181 (+ 565us) server_negotiation.cc:299] Negotiation successful
0806 22:37:46.772479 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":899,"thread_start_us":719,"threads_started":1}
I20260806 22:37:46.773478 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:46.746534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:52403)
0806 22:37:46.747570 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:46.747630 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:46.749724 (+ 2094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:46.750824 (+ 1100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:46.750836 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:46.751657 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:46.752472 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:46.752500 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.754302 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.754307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:46.756368 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:46.756386 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.757671 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.757683 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:46.758103 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:46.759197 (+ 1094us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:46.759226 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:46.762226 (+ 3000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:46.766965 (+ 4739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:46.766978 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:46.767054 (+ 76us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:46.767906 (+ 852us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:46.768576 (+ 670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:46.768580 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:46.768583 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:46.768867 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:46.769679 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:46.769693 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:46.770222 (+ 529us) client_negotiation.cc:769] Sending connection context
0806 22:37:46.770676 (+ 454us) client_negotiation.cc:240] Negotiation successful
0806 22:37:46.771456 (+ 780us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":447,"threads_started":1}
I20260806 22:37:46.775544 5258 heartbeater.cc:347] Connected to a master server at 127.15.198.190:38391
I20260806 22:37:46.776046 5258 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:46.777145 5258 heartbeater.cc:511] Master 127.15.198.190:38391 requested a full tablet report, sending...
I20260806 22:37:46.779364 4645 ts_manager.cc:194] Registered new tserver with Master: 353f00395f5a4b01b62aa5d8da2b12be (127.15.198.131:44049)
I20260806 22:37:46.780372 4645 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:52403
I20260806 22:37:46.790114 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:37:46.800136 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:46.791455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40900 (local address 127.15.198.190:38391)
0806 22:37:46.791788 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:46.791799 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:46.792320 (+ 521us) server_negotiation.cc:407] Connection header received
0806 22:37:46.792669 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:46.792674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:46.792953 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:46.793214 (+ 261us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:46.793776 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.794596 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.796713 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.797370 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.797513 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:46.798331 (+ 818us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:46.799217 (+ 886us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:46.799388 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:46.799618 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 22:37:46.799724 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 22:37:46.801578 4645 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:40900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:37:46.810762 4645 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:46.867349 5279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:46.831522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:44049 (local address 127.0.0.1:51634)
0806 22:37:46.843624 (+ 12102us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:46.843653 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:46.843797 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:46.856822 (+ 13025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:46.856833 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:46.856878 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:46.857517 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:46.857538 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.859791 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.859798 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:46.861941 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:46.861961 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.865887 (+ 3926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.865898 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:46.866125 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:46.866165 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 22:37:46.866363 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 22:37:46.866649 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11937,"mutex_wait_us":113,"thread_start_us":23484,"threads_started":1}
I20260806 22:37:46.867424 5273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:46.834176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:43697 (local address 127.0.0.1:47844)
0806 22:37:46.855443 (+ 21267us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:46.855476 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:46.855686 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:46.859099 (+ 3413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:46.859104 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:46.859131 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:46.859515 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:46.859525 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.861423 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.861430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:46.863355 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:46.863377 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.865748 (+ 2371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.865759 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:46.866028 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:46.866168 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 22:37:46.866323 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 22:37:46.866633 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21026,"mutex_wait_us":35,"thread_start_us":8024,"threads_started":1}
I20260806 22:37:46.868499 5280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:46.831227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:38563 (local address 127.0.0.1:36068)
0806 22:37:46.842990 (+ 11763us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:46.843125 (+ 135us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:46.843527 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:46.858359 (+ 14832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:46.858364 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:46.858396 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:46.858852 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:46.858864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.863311 (+ 4447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.863318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:46.865331 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:46.865374 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.867763 (+ 2389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.867771 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:46.867989 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:46.868028 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 22:37:46.868111 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:37:46.868230 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11257,"thread_start_us":17131,"threads_started":1}
I20260806 22:37:46.868613 5276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:46.832360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47844 (local address 127.15.198.130:43697)
0806 22:37:46.856363 (+ 24003us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:46.856374 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:46.856431 (+ 57us) server_negotiation.cc:407] Connection header received
0806 22:37:46.856578 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:46.856592 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:46.856903 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:46.859788 (+ 2885us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:46.860166 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.861232 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.863586 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.865246 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.865460 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:46.865679 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:46.867279 (+ 1600us) server_negotiation.cc:299] Negotiation successful
0806 22:37:46.868311 (+ 1032us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23810,"thread_start_us":23562,"threads_started":1}
I20260806 22:37:46.868970 5278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:46.833241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51634 (local address 127.15.198.131:44049)
0806 22:37:46.855941 (+ 22700us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:46.855951 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:46.856001 (+ 50us) server_negotiation.cc:407] Connection header received
0806 22:37:46.856131 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:46.856139 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:46.856454 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:46.856784 (+ 330us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:46.857779 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.859526 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.862174 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.864059 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.864282 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:46.864536 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:46.866462 (+ 1926us) server_negotiation.cc:299] Negotiation successful
0806 22:37:46.866704 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22459,"thread_start_us":23931,"threads_started":1}
I20260806 22:37:46.869874 5277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:46.832469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36068 (local address 127.15.198.129:38563)
0806 22:37:46.857775 (+ 25306us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:46.857783 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:46.857811 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:37:46.857914 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:46.857920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:46.858129 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:46.858381 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:46.861519 (+ 3138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.863089 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.865610 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.867428 (+ 1818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.868987 (+ 1559us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:46.869230 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:46.869311 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 22:37:46.869517 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24988,"thread_start_us":24656,"threads_started":1}
I20260806 22:37:46.873884 5168 tablet_service.cc:1467] Processing CreateTablet for tablet 500b07d80ad1453e961d7b2f16087f8e (DEFAULT_TABLE table=test-table [id=eecdcc34783e484c9164835cda7707ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:46.873888 4976 tablet_service.cc:1467] Processing CreateTablet for tablet 500b07d80ad1453e961d7b2f16087f8e (DEFAULT_TABLE table=test-table [id=eecdcc34783e484c9164835cda7707ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:46.875643 4976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 500b07d80ad1453e961d7b2f16087f8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:46.875674 5168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 500b07d80ad1453e961d7b2f16087f8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:46.878418 4804 tablet_service.cc:1467] Processing CreateTablet for tablet 500b07d80ad1453e961d7b2f16087f8e (DEFAULT_TABLE table=test-table [id=eecdcc34783e484c9164835cda7707ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:46.880863 4804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 500b07d80ad1453e961d7b2f16087f8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:46.892419 5281 tablet_bootstrap.cc:492] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417: Bootstrap starting.
I20260806 22:37:46.896030 5282 tablet_bootstrap.cc:492] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be: Bootstrap starting.
I20260806 22:37:46.897691 5281 tablet_bootstrap.cc:654] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:46.899605 5281 log.cc:824] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:46.901170 5282 tablet_bootstrap.cc:654] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:46.903430 5282 log.cc:824] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:46.906339 5285 tablet_bootstrap.cc:492] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b: Bootstrap starting.
I20260806 22:37:46.907624 5281 tablet_bootstrap.cc:492] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417: No bootstrap required, opened a new log
I20260806 22:37:46.908046 5281 ts_tablet_manager.cc:1397] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.004s
I20260806 22:37:46.908304 5282 tablet_bootstrap.cc:492] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be: No bootstrap required, opened a new log
I20260806 22:37:46.908740 5282 ts_tablet_manager.cc:1397] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be: Time spent bootstrapping tablet: real 0.013s user 0.012s sys 0.000s
I20260806 22:37:46.910656 5285 tablet_bootstrap.cc:654] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:46.912273 5285 log.cc:824] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:46.917304 5285 tablet_bootstrap.cc:492] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b: No bootstrap required, opened a new log
I20260806 22:37:46.917755 5285 ts_tablet_manager.cc:1397] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.004s
I20260806 22:37:46.921767 5282 raft_consensus.cc:348] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } }
I20260806 22:37:46.922053 5281 raft_consensus.cc:348] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } }
I20260806 22:37:46.922510 5282 raft_consensus.cc:374] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:46.922533 5281 raft_consensus.cc:374] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:46.922652 5282 raft_consensus.cc:729] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353f00395f5a4b01b62aa5d8da2b12be, State: Initialized, Role: FOLLOWER
I20260806 22:37:46.922647 5281 raft_consensus.cc:729] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5bb01607ba841f3bac49253ca74a417, State: Initialized, Role: FOLLOWER
I20260806 22:37:46.923676 5282 consensus_queue.cc:260] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } }
I20260806 22:37:46.923676 5281 consensus_queue.cc:260] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } }
I20260806 22:37:46.925444 5258 heartbeater.cc:503] Master 127.15.198.190:38391 was elected leader, sending a full tablet report...
I20260806 22:37:46.926764 5282 ts_tablet_manager.cc:1428] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be: Time spent starting tablet: real 0.018s user 0.008s sys 0.008s
I20260806 22:37:46.928061 5068 heartbeater.cc:503] Master 127.15.198.190:38391 was elected leader, sending a full tablet report...
I20260806 22:37:46.927964 5285 raft_consensus.cc:348] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } }
I20260806 22:37:46.928571 5285 raft_consensus.cc:374] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:46.928699 5285 raft_consensus.cc:729] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ac48cc3c2f3475a99fb6a8cc5e3727b, State: Initialized, Role: FOLLOWER
I20260806 22:37:46.928987 5281 ts_tablet_manager.cc:1428] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417: Time spent starting tablet: real 0.021s user 0.018s sys 0.001s
I20260806 22:37:46.929543 5285 consensus_queue.cc:260] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } }
I20260806 22:37:46.932088 5285 ts_tablet_manager.cc:1428] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
W20260806 22:37:46.978864 4884 tablet.cc:2369] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:37:46.994365 5259 tablet.cc:2369] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:37:47.120711 5069 tablet.cc:2369] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:37:47.347196 5289 raft_consensus.cc:482] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:47.347736 5289 raft_consensus.cc:504] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } }
I20260806 22:37:47.350596 5289 leader_election.cc:288] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ac48cc3c2f3475a99fb6a8cc5e3727b (127.15.198.129:38563), 353f00395f5a4b01b62aa5d8da2b12be (127.15.198.131:44049)
I20260806 22:37:47.359069 5278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.351346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:59201 (local address 127.15.198.131:44049)
0806 22:37:47.351711 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.351722 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.352674 (+ 952us) server_negotiation.cc:407] Connection header received
0806 22:37:47.352914 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.352919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.352996 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.353117 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:47.353795 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.354664 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.356893 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.357814 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.358057 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.358168 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.358534 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.358700 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260806 22:37:47.359113 5277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.351358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:50121 (local address 127.15.198.129:38563)
0806 22:37:47.351715 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.351725 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.352610 (+ 885us) server_negotiation.cc:407] Connection header received
0806 22:37:47.352941 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.352946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.353023 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.353132 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:47.353858 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.354697 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.356924 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.357862 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.358090 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.358202 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.358568 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.358741 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260806 22:37:47.359462 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.351293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:44049 (local address 127.15.198.130:59201)
0806 22:37:47.352486 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.352502 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.352703 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.353133 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.353138 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.353163 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:47.353616 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.353626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.354820 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.354826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.356683 (+ 1857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.356701 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.358130 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.358137 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.358331 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.358372 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.358493 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.358672 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1086,"mutex_wait_us":7,"thread_start_us":808,"threads_started":1}
I20260806 22:37:47.359663 5293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.351173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:38563 (local address 127.15.198.130:50121)
0806 22:37:47.352438 (+ 1265us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.352459 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.352690 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.353153 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.353158 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.353181 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:47.353695 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.353704 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.354845 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.354852 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.356702 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.356716 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.358189 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.358195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.358382 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.358407 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.358525 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.358674 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1095,"mutex_wait_us":28,"thread_start_us":802,"threads_started":1}
I20260806 22:37:47.360826 5194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "500b07d80ad1453e961d7b2f16087f8e" candidate_uuid: "d5bb01607ba841f3bac49253ca74a417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353f00395f5a4b01b62aa5d8da2b12be" is_pre_election: true
I20260806 22:37:47.360881 4826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "500b07d80ad1453e961d7b2f16087f8e" candidate_uuid: "d5bb01607ba841f3bac49253ca74a417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" is_pre_election: true
I20260806 22:37:47.361521 5194 raft_consensus.cc:2454] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bb01607ba841f3bac49253ca74a417 in term 0.
I20260806 22:37:47.361519 4826 raft_consensus.cc:2454] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bb01607ba841f3bac49253ca74a417 in term 0.
I20260806 22:37:47.361758 5288 raft_consensus.cc:482] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:47.362082 5288 raft_consensus.cc:504] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } }
I20260806 22:37:47.362429 4907 leader_election.cc:302] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ac48cc3c2f3475a99fb6a8cc5e3727b, d5bb01607ba841f3bac49253ca74a417; no voters:
I20260806 22:37:47.363059 5289 raft_consensus.cc:2781] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:47.363265 5289 raft_consensus.cc:482] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:47.363413 5289 raft_consensus.cc:3037] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:47.364717 5288 leader_election.cc:288] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5bb01607ba841f3bac49253ca74a417 (127.15.198.130:43697), 1ac48cc3c2f3475a99fb6a8cc5e3727b (127.15.198.129:38563)
I20260806 22:37:47.368590 5289 raft_consensus.cc:504] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } }
I20260806 22:37:47.369805 5289 leader_election.cc:288] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [CANDIDATE]: Term 1 election: Requested vote from peers 1ac48cc3c2f3475a99fb6a8cc5e3727b (127.15.198.129:38563), 353f00395f5a4b01b62aa5d8da2b12be (127.15.198.131:44049)
I20260806 22:37:47.370682 4826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "500b07d80ad1453e961d7b2f16087f8e" candidate_uuid: "d5bb01607ba841f3bac49253ca74a417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b"
I20260806 22:37:47.370705 5194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "500b07d80ad1453e961d7b2f16087f8e" candidate_uuid: "d5bb01607ba841f3bac49253ca74a417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353f00395f5a4b01b62aa5d8da2b12be"
I20260806 22:37:47.371044 4826 raft_consensus.cc:3037] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:47.371075 5194 raft_consensus.cc:3037] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:47.375298 4826 raft_consensus.cc:2454] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5bb01607ba841f3bac49253ca74a417 in term 1.
I20260806 22:37:47.375325 5194 raft_consensus.cc:2454] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5bb01607ba841f3bac49253ca74a417 in term 1.
I20260806 22:37:47.375686 5276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.366110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:56703 (local address 127.15.198.130:43697)
0806 22:37:47.366374 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.366381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.366403 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:37:47.366482 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.366488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.366572 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.366667 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:47.367650 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.369009 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.372777 (+ 3768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.374194 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.374357 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.374464 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.375142 (+ 678us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.375339 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 22:37:47.376019 4907 leader_election.cc:302] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ac48cc3c2f3475a99fb6a8cc5e3727b, d5bb01607ba841f3bac49253ca74a417; no voters:
I20260806 22:37:47.376684 5297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.365101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:38563 (local address 127.15.198.131:55999)
0806 22:37:47.367082 (+ 1981us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.367108 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.367270 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.367707 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.367714 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.367750 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:47.368183 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.368196 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.369809 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.369820 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.372921 (+ 3101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.372936 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.375322 (+ 2386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.375328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.375688 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.375733 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.375872 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.376051 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1805,"mutex_wait_us":54,"thread_start_us":1602,"threads_started":1}
I20260806 22:37:47.376871 5277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.365697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:55999 (local address 127.15.198.129:38563)
0806 22:37:47.367317 (+ 1620us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.367325 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.367349 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:37:47.367452 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.367457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.367557 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.367690 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:47.368411 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.369596 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.373167 (+ 3571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.374713 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.374900 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.375059 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.376349 (+ 1290us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.376545 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1404}
I20260806 22:37:47.376734 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.365033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:43697 (local address 127.15.198.131:56703)
0806 22:37:47.365747 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.365771 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.365973 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.366888 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.366893 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.366937 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:47.367460 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.367474 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.369180 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.369188 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.372461 (+ 3273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.372487 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.374630 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.374636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.374851 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.374909 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.375919 (+ 1010us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.376125 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":120,"thread_start_us":1168,"threads_started":1}
I20260806 22:37:47.377243 5289 raft_consensus.cc:2781] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:47.377938 4825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "500b07d80ad1453e961d7b2f16087f8e" candidate_uuid: "353f00395f5a4b01b62aa5d8da2b12be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" is_pre_election: true
I20260806 22:37:47.378525 4825 raft_consensus.cc:2379] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 353f00395f5a4b01b62aa5d8da2b12be in current term 1: Already voted for candidate d5bb01607ba841f3bac49253ca74a417 in this term.
I20260806 22:37:47.378630 5289 raft_consensus.cc:686] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [term 1 LEADER]: Becoming Leader. State: Replica: d5bb01607ba841f3bac49253ca74a417, State: Running, Role: LEADER
I20260806 22:37:47.378625 5003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "500b07d80ad1453e961d7b2f16087f8e" candidate_uuid: "353f00395f5a4b01b62aa5d8da2b12be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bb01607ba841f3bac49253ca74a417" is_pre_election: true
I20260806 22:37:47.379415 5289 consensus_queue.cc:237] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } } peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } }
I20260806 22:37:47.380067 5096 leader_election.cc:302] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 353f00395f5a4b01b62aa5d8da2b12be; no voters: 1ac48cc3c2f3475a99fb6a8cc5e3727b, d5bb01607ba841f3bac49253ca74a417
I20260806 22:37:47.380599 5288 raft_consensus.cc:2726] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 22:37:47.385524 4645 catalog_manager.cc:5495] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5bb01607ba841f3bac49253ca74a417 (127.15.198.130). New cstate: current_term: 1 leader_uuid: "d5bb01607ba841f3bac49253ca74a417" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bb01607ba841f3bac49253ca74a417" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 43697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 } health_report { overall_health: UNKNOWN } } }
I20260806 22:37:47.444520 5276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.436645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47848 (local address 127.15.198.130:43697)
0806 22:37:47.436977 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.436988 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.437425 (+ 437us) server_negotiation.cc:407] Connection header received
0806 22:37:47.437733 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.437738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.437876 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.438051 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:47.438627 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.439648 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.441380 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.441880 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.441996 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.442684 (+ 688us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:47.443584 (+ 900us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:47.443735 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.443924 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.444055 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 22:37:47.456480 5194 raft_consensus.cc:1261] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be [term 1 FOLLOWER]: Refusing update from remote peer d5bb01607ba841f3bac49253ca74a417: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:47.456508 4825 raft_consensus.cc:1261] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 1 FOLLOWER]: Refusing update from remote peer d5bb01607ba841f3bac49253ca74a417: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:47.457947 5298 consensus_queue.cc:1035] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "353f00395f5a4b01b62aa5d8da2b12be" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:47.458488 5289 consensus_queue.cc:1035] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:47.477165 5307 mvcc.cc:205] Tried to move back new op lower bound from 7315684833086427136 to 7315684832791183360. Current Snapshot: MvccSnapshot[applied={T|T < 7315684833086427136}]
I20260806 22:37:47.478091 5310 mvcc.cc:205] Tried to move back new op lower bound from 7315684833086427136 to 7315684832791183360. Current Snapshot: MvccSnapshot[applied={T|T < 7315684833086427136}]
I20260806 22:37:47.513378 4645 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:40900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:47.513850 4645 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:40900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:47.519562 4645 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54: Sending DeleteTablet for 3 replicas of tablet 500b07d80ad1453e961d7b2f16087f8e
I20260806 22:37:47.521548 4976 tablet_service.cc:1514] Processing DeleteTablet for tablet 500b07d80ad1453e961d7b2f16087f8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:47 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:47844
I20260806 22:37:47.521548 4804 tablet_service.cc:1514] Processing DeleteTablet for tablet 500b07d80ad1453e961d7b2f16087f8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:47 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:36068
I20260806 22:37:47.521544 5168 tablet_service.cc:1514] Processing DeleteTablet for tablet 500b07d80ad1453e961d7b2f16087f8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:47 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:51634
I20260806 22:37:47.523617 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 4689
I20260806 22:37:47.549510 5320 tablet_replica.cc:330] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b: stopping tablet replica
I20260806 22:37:47.550432 5320 raft_consensus.cc:2229] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:47.553301 5325 tablet_replica.cc:330] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417: stopping tablet replica
I20260806 22:37:47.553934 5320 raft_consensus.cc:2258] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:47.554780 5325 raft_consensus.cc:2229] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:37:47.555986 5325 raft_consensus.cc:2258] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:47.556365 5324 tablet_replica.cc:330] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be: stopping tablet replica
I20260806 22:37:47.557085 5320 ts_tablet_manager.cc:1905] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:47.557119 5324 raft_consensus.cc:2229] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:47.560084 5324 raft_consensus.cc:2258] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:47.560319 5325 ts_tablet_manager.cc:1905] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:47.564368 5324 ts_tablet_manager.cc:1905] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:47.568902 5320 ts_tablet_manager.cc:1918] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:47.569335 5320 log.cc:1196] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/500b07d80ad1453e961d7b2f16087f8e
I20260806 22:37:47.570134 5320 ts_tablet_manager.cc:1939] T 500b07d80ad1453e961d7b2f16087f8e P 1ac48cc3c2f3475a99fb6a8cc5e3727b: Deleting consensus metadata
I20260806 22:37:47.572201 5277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.524859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36076 (local address 127.15.198.129:38563)
0806 22:37:47.525178 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.525189 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.547940 (+ 22751us) server_negotiation.cc:407] Connection header received
0806 22:37:47.548233 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.548244 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.548406 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.548588 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.552174 (+ 3586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.553741 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.556315 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.557083 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.557292 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.562095 (+ 4803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.562122 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.562147 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.562207 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.567368 (+ 5161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.568923 (+ 1555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.568939 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.568944 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.569200 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.569969 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.569976 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.569979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.570675 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.571140 (+ 465us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.571458 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.571778 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 22:37:47.573093 4620 catalog_manager.cc:4841] TS 1ac48cc3c2f3475a99fb6a8cc5e3727b (127.15.198.129:38563): tablet 500b07d80ad1453e961d7b2f16087f8e (table test-table [id=eecdcc34783e484c9164835cda7707ce]) successfully deleted
I20260806 22:37:47.573684 4877 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:47.578455 5324 ts_tablet_manager.cc:1918] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:47.579092 5324 log.cc:1196] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/500b07d80ad1453e961d7b2f16087f8e
I20260806 22:37:47.580179 5324 ts_tablet_manager.cc:1939] T 500b07d80ad1453e961d7b2f16087f8e P 353f00395f5a4b01b62aa5d8da2b12be: Deleting consensus metadata
I20260806 22:37:47.580610 5325 ts_tablet_manager.cc:1918] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:47.580950 5325 log.cc:1196] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/500b07d80ad1453e961d7b2f16087f8e
I20260806 22:37:47.581758 5325 ts_tablet_manager.cc:1939] T 500b07d80ad1453e961d7b2f16087f8e P d5bb01607ba841f3bac49253ca74a417: Deleting consensus metadata
I20260806 22:37:47.583909 4621 catalog_manager.cc:4841] TS 353f00395f5a4b01b62aa5d8da2b12be (127.15.198.131:44049): tablet 500b07d80ad1453e961d7b2f16087f8e (table test-table [id=eecdcc34783e484c9164835cda7707ce]) successfully deleted
I20260806 22:37:47.584095 4622 catalog_manager.cc:4841] TS d5bb01607ba841f3bac49253ca74a417 (127.15.198.130:43697): tablet 500b07d80ad1453e961d7b2f16087f8e (table test-table [id=eecdcc34783e484c9164835cda7707ce]) successfully deleted
I20260806 22:37:47.691417 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 4689
I20260806 22:37:47.743645 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 4887
I20260806 22:37:47.767421 5276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.744967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47860 (local address 127.15.198.130:43697)
0806 22:37:47.745231 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.745240 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.745269 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:37:47.745412 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.745418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.745530 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.745664 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.748113 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.749396 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.751655 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.752408 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.752604 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.757150 (+ 4546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.757180 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.757209 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.757287 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.762771 (+ 5484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.764114 (+ 1343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.764134 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.764138 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.764370 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.764935 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.764940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.764944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.765608 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.766079 (+ 471us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.766466 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.766794 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 22:37:47.768610 5060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:47.916918 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 4887
I20260806 22:37:47.955416 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 5072
I20260806 22:37:47.974547 5329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.956458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51650 (local address 127.15.198.131:44049)
0806 22:37:47.957145 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.957150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.957172 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:37:47.957248 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.957252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.957327 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.957458 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.959379 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.960201 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.961864 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.962420 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.962567 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.966601 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.966625 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.966650 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.966710 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.970874 (+ 4164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.972217 (+ 1343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.972228 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.972230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.972409 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.972837 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.972840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.972842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.973344 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.973643 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.973956 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.974207 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":411,"threads_started":1}
I20260806 22:37:47.975667 5252 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:48.081102 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 5072
I20260806 22:37:48.121325 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 4596
I20260806 22:37:48.123167 4682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:37:48.260288 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 4596
I20260806 22:37:48.299170 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:38391
--webserver_interface=127.15.198.190
--webserver_port=36783
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:38391
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:48.754421 5334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:48.754865 5334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:48.754979 5334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:48.764148 5334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:48.764324 5334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:48.764379 5334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:48.764443 5334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:48.764492 5334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:48.776361 5334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:38391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:38391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=36783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:48.778265 5334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:48.780184 5334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:48.795836 5343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:48.796005 5342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:48.795728 5345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:48.797632 5334 server_base.cc:1034] running on GCE node
I20260806 22:37:48.798877 5334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:48.801209 5334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:48.802527 5334 hybrid_clock.cc:648] HybridClock initialized: now 1786055868802460 us; error 49 us; skew 500 ppm
I20260806 22:37:48.803160 5334 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:48 dist-test-slave-h3ft krb5kdc[3810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055868, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:48.812270 5334 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:37:48.814580 5334 webserver.cc:460] Webserver started at http://127.15.198.190:36783/ using document root <none> and password file <none>
I20260806 22:37:48.815652 5334 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:48.815824 5334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:48.823536 5334 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 22:37:48.827491 5353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:48.829399 5334 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:37:48.829654 5334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "73dc724fc1664869a3e9b0b34b949f54"
format_stamp: "Formatted at 2026-08-06 22:37:40 on dist-test-slave-h3ft"
server_key: "b0747b1c7a162e55e1a256403509b316"
server_key_iv: "06735747abe8e9d65283d6fa655d8d79"
server_key_version: "encryptionkey@0"
I20260806 22:37:48.830965 5334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:48.881691 5334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:48.887835 5334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:48.888248 5334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:48.913177 5334 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:38391
I20260806 22:37:48.913198 5432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:38391 every 8 connection(s)
I20260806 22:37:48.915405 5334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:48.924840 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 5334
I20260806 22:37:48.925036 5435 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:37:48.926030 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:38563
--local_ip_for_outbound_sockets=127.15.198.129
--tserver_master_addrs=127.15.198.190:38391
--webserver_port=42073
--webserver_interface=127.15.198.129
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:37:48.929693 5435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54: Bootstrap starting.
I20260806 22:37:48.956938 5435 log.cc:824] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:48.977640 5435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:37:48.978452 5435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54: Bootstrap complete.
I20260806 22:37:48.989951 5435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } }
I20260806 22:37:48.991158 5435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73dc724fc1664869a3e9b0b34b949f54, State: Initialized, Role: FOLLOWER
I20260806 22:37:48.992235 5435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } }
I20260806 22:37:48.992527 5435 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:48.992686 5435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:48.992925 5435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:37:48.997769 5435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } }
I20260806 22:37:48.998468 5435 leader_election.cc:302] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73dc724fc1664869a3e9b0b34b949f54; no voters:
I20260806 22:37:48.999365 5435 leader_election.cc:288] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 22:37:48.999572 5441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:37:49.001209 5441 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [term 3 LEADER]: Becoming Leader. State: Replica: 73dc724fc1664869a3e9b0b34b949f54, State: Running, Role: LEADER
I20260806 22:37:49.002105 5441 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } }
I20260806 22:37:49.002658 5435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:49.005661 5444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73dc724fc1664869a3e9b0b34b949f54. Latest consensus state: current_term: 3 leader_uuid: "73dc724fc1664869a3e9b0b34b949f54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } } }
I20260806 22:37:49.005730 5442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "73dc724fc1664869a3e9b0b34b949f54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73dc724fc1664869a3e9b0b34b949f54" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 38391 } } }
I20260806 22:37:49.006301 5444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:49.006307 5442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73dc724fc1664869a3e9b0b34b949f54 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:49.008512 5449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:49.029624 5449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:49.031672 5449 catalog_manager.cc:1259] Loaded cluster ID: 2b2d584aa03a466ba0e2824592d0ddf7
I20260806 22:37:49.031949 5449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 22:37:49.034611 5449 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f36d43159ad google::LogMessage::Fail() at ??:0
@ 0x7f36d431888a google::LogMessage::SendToLog() at ??:0
@ 0x7f36d43154de google::LogMessage::Flush() at ??:0
@ 0x7f36d4316a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f36e9d48512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f36e9d45976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f36e9dcd610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f36e9dcd439 std::_Function_handler<>::_M_invoke() at ??:0
I20260806 22:37:49.096417 5476 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f36d6f61345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f36d6f6491c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f36d6f64799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f36d6f3feaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f36da2196db start_thread at ??:0
@ 0x7f36d31af61f clone at ??:0
W20260806 22:37:49.444355 5438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:49.444824 5438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:49.445029 5438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:49.453444 5438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:49.453661 5438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:49.453882 5438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:37:49.465444 5438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:38563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=42073
--tserver_master_addrs=127.15.198.190:38391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:49.467379 5438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:49.469292 5438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:49.486524 5502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:49.486501 5503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:49.486459 5508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:49.488001 5438 server_base.cc:1034] running on GCE node
I20260806 22:37:49.489001 5438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:49.490350 5438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:49.491616 5438 hybrid_clock.cc:648] HybridClock initialized: now 1786055869491547 us; error 58 us; skew 500 ppm
I20260806 22:37:49.492159 5438 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:49 dist-test-slave-h3ft krb5kdc[3810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055869, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:49.496829 5438 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:37:49.499508 5438 webserver.cc:460] Webserver started at http://127.15.198.129:42073/ using document root <none> and password file <none>
I20260806 22:37:49.500563 5438 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:49.500739 5438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:49.508405 5438 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 22:37:49.512337 5516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:49.514511 5438 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 22:37:49.514792 5438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "1ac48cc3c2f3475a99fb6a8cc5e3727b"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-h3ft"
server_key: "34d906cd82fafd7e617a6cfc84151471"
server_key_iv: "d2ed13d13bd2cbdc13962e3b681347e7"
server_key_version: "encryptionkey@0"
I20260806 22:37:49.516239 5438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:49.547804 5438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:49.553668 5438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:49.554049 5438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:49.555445 5438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:49.557360 5438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:49.557484 5438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:49.557658 5438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:49.557766 5438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:49.600998 5438 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:38563
I20260806 22:37:49.601123 5665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:38563 every 8 connection(s)
I20260806 22:37:49.602977 5438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:37:49.609985 5672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:49.607444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:47425)
0806 22:37:49.608461 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:49.608755 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":403,"threads_started":1}
W20260806 22:37:49.610984 5667 heartbeater.cc:644] Failed to heartbeat to 127.15.198.190:38391 (0 consecutive failures): Network error: Failed to ping master at 127.15.198.190:38391: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
I20260806 22:37:49.612366 5672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:49.611819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:32877)
0806 22:37:49.612105 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:49.612202 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260806 22:37:49.612462 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 5438
I20260806 22:37:49.613510 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:43697
--local_ip_for_outbound_sockets=127.15.198.130
--tserver_master_addrs=127.15.198.190:38391
--webserver_port=33341
--webserver_interface=127.15.198.130
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:37:49.613704 5672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:49.613202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:47919)
0806 22:37:49.613398 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:49.613503 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 22:37:49.614005 5667 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:37:50.064613 5673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:50.065066 5673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:50.065183 5673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:50.073796 5673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:50.073983 5673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:50.074124 5673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:37:50.085848 5673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:43697
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=33341
--tserver_master_addrs=127.15.198.190:38391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:50.087776 5673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:50.089704 5673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:50.108242 5678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:50.108508 5679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:50.108511 5681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:50.110078 5673 server_base.cc:1034] running on GCE node
I20260806 22:37:50.111157 5673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:50.112641 5673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:50.113978 5673 hybrid_clock.cc:648] HybridClock initialized: now 1786055870113887 us; error 79 us; skew 500 ppm
I20260806 22:37:50.114506 5673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:50 dist-test-slave-h3ft krb5kdc[3810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055870, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:50.119172 5673 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:37:50.122161 5673 webserver.cc:460] Webserver started at http://127.15.198.130:33341/ using document root <none> and password file <none>
I20260806 22:37:50.123251 5673 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:50.123441 5673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:50.131255 5673 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 22:37:50.135445 5691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:50.137542 5673 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:37:50.137816 5673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "d5bb01607ba841f3bac49253ca74a417"
format_stamp: "Formatted at 2026-08-06 22:37:42 on dist-test-slave-h3ft"
server_key: "b2fa861ed409729b29ae4a5cbde2761d"
server_key_iv: "1030e928d6cc38feba095f9920f08c79"
server_key_version: "encryptionkey@0"
I20260806 22:37:50.139233 5673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:50.164240 5673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:50.170275 5673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:50.170694 5673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:50.172210 5673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:50.174101 5673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:50.174250 5673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:50.174423 5673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:50.174525 5673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:50.215713 5673 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:43697
I20260806 22:37:50.215768 5844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:43697 every 8 connection(s)
I20260806 22:37:50.217767 5673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:37:50.224658 5847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.222088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:51841)
0806 22:37:50.223183 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.223477 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":392,"threads_started":1}
W20260806 22:37:50.225526 5845 heartbeater.cc:644] Failed to heartbeat to 127.15.198.190:38391 (0 consecutive failures): Network error: Failed to ping master at 127.15.198.190:38391: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
I20260806 22:37:50.226634 5847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.226219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:41471)
0806 22:37:50.226407 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.226504 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 22:37:50.227650 5847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.227269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:33291)
0806 22:37:50.227416 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.227522 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 22:37:50.227900 5845 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:37:50.228006 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 5673
I20260806 22:37:50.228788 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:44049
--local_ip_for_outbound_sockets=127.15.198.131
--tserver_master_addrs=127.15.198.190:38391
--webserver_port=40295
--webserver_interface=127.15.198.131
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:37:50.616312 5853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.615138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:43959)
0806 22:37:50.615824 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.615991 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":363,"threads_started":1}
W20260806 22:37:50.679183 5852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:50.679608 5852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:50.679811 5852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:50.688532 5852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:50.688714 5852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:50.688920 5852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:37:50.701172 5852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:44049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=40295
--tserver_master_addrs=127.15.198.190:38391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:37:50.703293 5852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:50.705075 5852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:50.722218 5858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:50.722185 5861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:50.722810 5859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:50.723426 5852 server_base.cc:1034] running on GCE node
I20260806 22:37:50.724229 5852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:50.725479 5852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:50.726717 5852 hybrid_clock.cc:648] HybridClock initialized: now 1786055870726649 us; error 52 us; skew 500 ppm
I20260806 22:37:50.727213 5852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:37:50 dist-test-slave-h3ft krb5kdc[3810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055870, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:50.731395 5852 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:37:50.733378 5852 webserver.cc:460] Webserver started at http://127.15.198.131:40295/ using document root <none> and password file <none>
I20260806 22:37:50.734308 5852 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:50.734480 5852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:50.741534 5852 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:37:50.745271 5870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:50.747215 5852 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:37:50.747459 5852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "353f00395f5a4b01b62aa5d8da2b12be"
format_stamp: "Formatted at 2026-08-06 22:37:43 on dist-test-slave-h3ft"
server_key: "7880c83fc596fa97bca030ccadc83278"
server_key_iv: "e39c9835b1ce9df21ecfb9daedaea55d"
server_key_version: "encryptionkey@0"
I20260806 22:37:50.748804 5852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:50.775655 5852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:50.781855 5852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:50.782265 5852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:50.783679 5852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:50.785594 5852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:50.785746 5852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:50.785908 5852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:50.786011 5852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:50.829378 5852 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:44049
I20260806 22:37:50.829432 6028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:44049 every 8 connection(s)
I20260806 22:37:50.831601 5852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:37:50.833376 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 5852
I20260806 22:37:50.838634 6034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.836254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:54933)
0806 22:37:50.837183 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.837477 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":352,"threads_started":1}
W20260806 22:37:50.839551 6029 heartbeater.cc:644] Failed to heartbeat to 127.15.198.190:38391 (0 consecutive failures): Network error: Failed to ping master at 127.15.198.190:38391: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
I20260806 22:37:50.840687 6034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.840264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:49705)
0806 22:37:50.840454 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.840545 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 22:37:50.841681 6034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.841281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:43605)
0806 22:37:50.841468 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.841548 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260806 22:37:50.841920 6029 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:37:51.229867 6036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:51.228730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:33943)
0806 22:37:51.229333 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:51.229479 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":297,"threads_started":1}
I20260806 22:37:51.618717 6038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:51.617702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:45225)
0806 22:37:51.618349 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:51.618477 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":327,"threads_started":1}
I20260806 22:37:51.844194 6039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:51.842860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:45441)
0806 22:37:51.843595 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:51.843871 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":353,"threads_started":1}
I20260806 22:37:52.232471 6042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.231315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:53881)
0806 22:37:52.232049 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:52.232182 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":349,"threads_started":1}
I20260806 22:37:52.621366 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.620202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:57047)
0806 22:37:52.620837 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:52.621056 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":338,"threads_started":1}
I20260806 22:37:52.847054 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.845721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:44177)
0806 22:37:52.846531 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:52.846686 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":382,"threads_started":1}
I20260806 22:37:53.234884 6050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.233817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:59995)
0806 22:37:53.234468 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:53.234599 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":343,"threads_started":1}
I20260806 22:37:53.624022 6051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.622714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:51133)
0806 22:37:53.623650 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:53.623774 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":791,"thread_start_us":614,"threads_started":1}
I20260806 22:37:53.849583 6052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.848610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:58403)
0806 22:37:53.849215 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:53.849331 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":297,"threads_started":1}
I20260806 22:37:54.237282 6053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:54.236216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:52961)
0806 22:37:54.236922 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:54.237038 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":365,"threads_started":1}
I20260806 22:37:54.626304 6055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:54.625279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:48189)
0806 22:37:54.625923 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:54.626076 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":319,"threads_started":1}
I20260806 22:37:54.851963 6056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:54.850934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:52367)
0806 22:37:54.851617 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:54.851731 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":312,"threads_started":1}
I20260806 22:37:55.239789 6057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:55.238714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:39235)
0806 22:37:55.239462 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:55.239561 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":432,"threads_started":1}
I20260806 22:37:55.628979 6059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:55.627915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:42913)
0806 22:37:55.628640 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:55.628758 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":402,"threads_started":1}
I20260806 22:37:55.854722 6061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:55.853370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:36679)
0806 22:37:55.854286 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:55.854421 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":762,"thread_start_us":528,"threads_started":1}
I20260806 22:37:56.242265 6062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:56.241139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:40673)
0806 22:37:56.241893 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:56.242032 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":410,"threads_started":1}
I20260806 22:37:56.631412 6063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:56.630293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:56701)
0806 22:37:56.630969 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:56.631142 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":319,"threads_started":1}
I20260806 22:37:56.857343 6064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:56.856188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:35377)
0806 22:37:56.856944 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:56.857094 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":415,"threads_started":1}
I20260806 22:37:57.244920 6065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:57.243774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:51867)
0806 22:37:57.244532 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:57.244643 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":389,"threads_started":1}
I20260806 22:37:57.634032 6066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:57.632904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:57091)
0806 22:37:57.633678 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:57.633805 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":450,"threads_started":1}
I20260806 22:37:57.860014 6069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:57.858830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:53049)
0806 22:37:57.859578 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:57.859696 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":358,"threads_started":1}
I20260806 22:37:58.247380 6070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:58.246297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:40501)
0806 22:37:58.246953 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:58.247153 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":323,"threads_started":1}
I20260806 22:37:58.636427 6072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:58.635410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:47905)
0806 22:37:58.636063 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:58.636196 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":316,"threads_started":1}
I20260806 22:37:58.862299 6073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:58.861357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:41439)
0806 22:37:58.861950 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:58.862067 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":298,"threads_started":1}
I20260806 22:37:59.249673 6076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:59.248621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:57487)
0806 22:37:59.249288 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:59.249411 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":310,"threads_started":1}
I20260806 22:37:59.638657 6078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:59.637734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:45581)
0806 22:37:59.638321 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:59.638424 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":323,"threads_started":1}
I20260806 22:37:59.864558 6079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:59.863559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:38501)
0806 22:37:59.864194 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:59.864308 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":307,"threads_started":1}
I20260806 22:38:00.252234 6080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:00.251222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:57927)
0806 22:38:00.251888 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:00.252010 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":360,"threads_started":1}
I20260806 22:38:00.641085 6081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:00.640023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:36051)
0806 22:38:00.640674 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:00.640813 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":331,"threads_started":1}
I20260806 22:38:00.867139 6082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:00.865922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:37951)
0806 22:38:00.866692 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:00.866816 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":434,"threads_started":1}
I20260806 22:38:01.255200 6083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:01.253914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:33243)
0806 22:38:01.254698 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:01.254848 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":429,"threads_started":1}
I20260806 22:38:01.644062 6086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:01.642774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:55385)
0806 22:38:01.643681 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:01.643839 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":513,"threads_started":1}
I20260806 22:38:01.869768 6087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:01.868563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:55743)
0806 22:38:01.869300 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:01.869459 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":462,"threads_started":1}
I20260806 22:38:02.258265 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:02.256861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:39525)
0806 22:38:02.257569 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:02.257694 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":378,"threads_started":1}
I20260806 22:38:02.646586 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:02.645469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:38607)
0806 22:38:02.646159 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:02.646316 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":354,"threads_started":1}
I20260806 22:38:02.872387 6090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:02.871467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:45803)
0806 22:38:02.872057 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:02.872173 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":304,"threads_started":1}
I20260806 22:38:03.260959 6091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:03.259829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:48645)
0806 22:38:03.260575 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:03.260699 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":405,"threads_started":1}
I20260806 22:38:03.649037 6094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:03.648006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:36161)
0806 22:38:03.648660 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:03.648784 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":306,"threads_started":1}
I20260806 22:38:03.874956 6096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:03.873859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:53247)
0806 22:38:03.874532 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:03.874665 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":337,"threads_started":1}
I20260806 22:38:04.264004 6097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:04.262619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:53805)
0806 22:38:04.263525 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:04.263675 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":509,"threads_started":1}
I20260806 22:38:04.652156 6099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:04.651112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:53519)
0806 22:38:04.651729 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:04.651882 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":330,"threads_started":1}
I20260806 22:38:04.877521 6100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:04.876432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:36631)
0806 22:38:04.877136 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:04.877258 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":342,"threads_started":1}
I20260806 22:38:05.267243 6102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:05.265974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.130:51707)
0806 22:38:05.266734 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:05.266906 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":415,"threads_started":1}
I20260806 22:38:05.655431 6103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:05.653709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.129:48239)
0806 22:38:05.654701 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:05.655159 (+ 458us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":830,"thread_start_us":404,"threads_started":1}
I20260806 22:38:05.842684 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 5438
I20260806 22:38:05.880842 6104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:05.844935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52586 (local address 127.15.198.129:38563)
0806 22:38:05.846032 (+ 1097us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:05.846057 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:05.846120 (+ 63us) server_negotiation.cc:407] Connection header received
0806 22:38:05.848497 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:05.848515 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:05.849207 (+ 692us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:05.849750 (+ 543us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:05.852321 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:05.854349 (+ 2028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:05.857076 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:05.858117 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:05.858535 (+ 418us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:05.869702 (+ 11167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:05.869716 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:05.869744 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:05.869824 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:05.874733 (+ 4909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:05.876193 (+ 1460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:05.876206 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:05.876222 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:05.876572 (+ 350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:05.877290 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:05.877296 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:05.877298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:05.878303 (+ 1005us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:05.878741 (+ 438us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:05.879673 (+ 932us) server_negotiation.cc:299] Negotiation successful
0806 22:38:05.880457 (+ 784us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":827,"thread_start_us":631,"threads_started":1}
I20260806 22:38:05.880883 6105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:05.879391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:38391 (local address 127.15.198.131:52171)
0806 22:38:05.880505 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:05.880663 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:38391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":960,"thread_start_us":594,"threads_started":1}
I20260806 22:38:05.883406 5661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:38:05.989006 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 5438
I20260806 22:38:06.026328 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 5673
I20260806 22:38:06.053471 6108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:06.027783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53402 (local address 127.15.198.130:43697)
0806 22:38:06.028454 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:06.028474 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:06.028567 (+ 93us) server_negotiation.cc:407] Connection header received
0806 22:38:06.030479 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:06.030494 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:06.031327 (+ 833us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:06.032000 (+ 673us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:06.034028 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:06.035966 (+ 1938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:06.037905 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:06.038980 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:06.039319 (+ 339us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:06.043515 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:06.043528 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:06.043555 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:06.043631 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:06.048035 (+ 4404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:06.049638 (+ 1603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:06.049656 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:06.049667 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:06.049977 (+ 310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:06.050518 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:06.050522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:06.050525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:06.051460 (+ 935us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:06.051782 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:06.052356 (+ 574us) server_negotiation.cc:299] Negotiation successful
0806 22:38:06.053023 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":315,"threads_started":1}
I20260806 22:38:06.055815 5840 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:38:06.156876 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 5673
I20260806 22:38:06.196193 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 5852
I20260806 22:38:06.222615 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:06.197596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38736 (local address 127.15.198.131:44049)
0806 22:38:06.198454 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:06.198476 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:06.198539 (+ 63us) server_negotiation.cc:407] Connection header received
0806 22:38:06.201000 (+ 2461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:06.201018 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:06.201750 (+ 732us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:06.202237 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:06.204862 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:06.206474 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:06.208371 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:06.209431 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:06.209639 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:06.213383 (+ 3744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:06.213398 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:06.213429 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:06.213508 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:06.217551 (+ 4043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:06.218740 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:06.218751 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:06.218760 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:06.219127 (+ 367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:06.219627 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:06.219632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:06.219635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:06.220525 (+ 890us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:06.220835 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:06.221451 (+ 616us) server_negotiation.cc:299] Negotiation successful
0806 22:38:06.222169 (+ 718us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":402,"threads_started":1}
I20260806 22:38:06.224852 6023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:38:06.325994 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 5852
2026-08-06T22:38:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (27932 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:38:06 dist-test-slave-h3ft krb5kdc[6115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:38:06 dist-test-slave-h3ft krb5kdc[6115](info): set up 2 sockets
Aug 06 22:38:06 dist-test-slave-h3ft krb5kdc[6115](info): commencing operation
krb5kdc: starting...
W20260806 22:38:07.828570 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.440s user 0.000s sys 0.065s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:38:08 dist-test-slave-h3ft krb5kdc[6115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:38:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:38:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:38:08.260406 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:33495
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:42223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:33495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:38:08.714876 6129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:08.715430 6129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:08.715798 6129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:08.724802 6129 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:38:08.725056 6129 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:38:08.725178 6129 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:38:08.725289 6129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:38:08.725420 6129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:08.725585 6129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:08.725684 6129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:38:08.725790 6129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:38:08.738255 6129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:42223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:33495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:33495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:38:08.740322 6129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:08.742218 6129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:08.758608 6137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:08.758702 6138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:08.758648 6141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:08.759675 6129 server_base.cc:1034] running on GCE node
I20260806 22:38:08.760807 6129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:08.762768 6129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:08.764079 6129 hybrid_clock.cc:648] HybridClock initialized: now 1786055888763978 us; error 64 us; skew 500 ppm
I20260806 22:38:08.764510 6129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:38:08 dist-test-slave-h3ft krb5kdc[6115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055888, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:38:08.768638 6129 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:38:08.770709 6129 webserver.cc:460] Webserver started at http://127.15.198.190:38655/ using document root <none> and password file <none>
I20260806 22:38:08.771716 6129 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:08.771827 6129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:08.772176 6129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:38:08.776257 6129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "c042426a636e498b9996a841635e7dff"
format_stamp: "Formatted at 2026-08-06 22:38:08 on dist-test-slave-h3ft"
server_key: "a548424a82b091aa7dba34c3524f8931"
server_key_iv: "1a1eb6d0b5b3265a545e1c8daa1b7d68"
server_key_version: "encryptionkey@0"
I20260806 22:38:08.777261 6129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "c042426a636e498b9996a841635e7dff"
format_stamp: "Formatted at 2026-08-06 22:38:08 on dist-test-slave-h3ft"
server_key: "a548424a82b091aa7dba34c3524f8931"
server_key_iv: "1a1eb6d0b5b3265a545e1c8daa1b7d68"
server_key_version: "encryptionkey@0"
I20260806 22:38:08.784565 6129 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260806 22:38:08.789479 6152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:08.791544 6129 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:38:08.791838 6129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "c042426a636e498b9996a841635e7dff"
format_stamp: "Formatted at 2026-08-06 22:38:08 on dist-test-slave-h3ft"
server_key: "a548424a82b091aa7dba34c3524f8931"
server_key_iv: "1a1eb6d0b5b3265a545e1c8daa1b7d68"
server_key_version: "encryptionkey@0"
I20260806 22:38:08.792160 6129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:08.811823 6129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:08.817535 6129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:08.817904 6129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:08.842314 6129 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:33495
I20260806 22:38:08.842344 6220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:33495 every 8 connection(s)
I20260806 22:38:08.844785 6129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:38:08.851524 6221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:38:08.854964 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 6129
I20260806 22:38:08.855293 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:38:08.856316 16154 external_mini_cluster.cc:1428] Setting key 8f626860a89abb8057901ee97865a31b
I20260806 22:38:08.867690 6221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Bootstrap starting.
I20260806 22:38:08.872200 6221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Neither blocks nor log segments found. Creating new log.
I20260806 22:38:08.873621 6221 log.cc:824] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Log is configured to *not* fsync() on all Append() calls
Aug 06 22:38:08 dist-test-slave-h3ft krb5kdc[6115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:38:08.878041 6221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: No bootstrap required, opened a new log
I20260806 22:38:08.885722 6225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:08.859315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59130 (local address 127.15.198.190:33495)
0806 22:38:08.860484 (+ 1169us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:08.860502 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:08.860561 (+ 59us) server_negotiation.cc:407] Connection header received
0806 22:38:08.862652 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:08.862683 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:08.863326 (+ 643us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:08.863835 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:08.865831 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:08.867501 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:08.869404 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:08.870280 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:08.870526 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:08.876148 (+ 5622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:08.876159 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:08.876181 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:08.876229 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:08.879985 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:08.880954 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:08.880963 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:08.880973 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:08.881209 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:08.881664 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:08.881676 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:08.881680 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:08.882441 (+ 761us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:08.882834 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:08.883435 (+ 601us) server_negotiation.cc:299] Negotiation successful
0806 22:38:08.884140 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":875,"thread_start_us":541,"threads_started":1}
I20260806 22:38:08.887813 6221 raft_consensus.cc:348] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } }
I20260806 22:38:08.888375 6221 raft_consensus.cc:374] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:38:08.888532 6221 raft_consensus.cc:729] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c042426a636e498b9996a841635e7dff, State: Initialized, Role: FOLLOWER
I20260806 22:38:08.889351 6221 consensus_queue.cc:260] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } }
I20260806 22:38:08.889698 6221 raft_consensus.cc:388] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:38:08.889848 6221 raft_consensus.cc:482] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:38:08.890080 6221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:38:08.892807 6221 raft_consensus.cc:504] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } }
I20260806 22:38:08.893457 6221 leader_election.cc:302] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c042426a636e498b9996a841635e7dff; no voters:
I20260806 22:38:08.894330 6221 leader_election.cc:288] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:38:08.894503 6227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:38:08.895862 6227 raft_consensus.cc:686] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 1 LEADER]: Becoming Leader. State: Replica: c042426a636e498b9996a841635e7dff, State: Running, Role: LEADER
I20260806 22:38:08.896557 6227 consensus_queue.cc:237] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } }
I20260806 22:38:08.897478 6221 sys_catalog.cc:564] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:38:08.903575 6229 sys_catalog.cc:455] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [sys.catalog]: SysCatalogTable state changed. Reason: New leader c042426a636e498b9996a841635e7dff. Latest consensus state: current_term: 1 leader_uuid: "c042426a636e498b9996a841635e7dff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } } }
I20260806 22:38:08.903878 6228 sys_catalog.cc:455] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c042426a636e498b9996a841635e7dff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } } }
I20260806 22:38:08.904240 6229 sys_catalog.cc:458] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [sys.catalog]: This master's current role is: LEADER
I20260806 22:38:08.904299 6228 sys_catalog.cc:458] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [sys.catalog]: This master's current role is: LEADER
I20260806 22:38:08.905927 6242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:38:08.913300 6242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:38:08.925451 6242 catalog_manager.cc:1347] Generated new cluster ID: 335287833ba049098ee01aa86dbdfa52
I20260806 22:38:08.925624 6242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:38:08.963979 6242 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:38:08.966495 6242 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:38:08.982362 6242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Generated new TSK 0
I20260806 22:38:08.984009 6242 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
I20260806 22:38:09.087633 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33495
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:42223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:38:09.533026 6259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:09.533444 6259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:09.533556 6259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:09.542089 6259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:09.542255 6259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:09.542392 6259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:38:09.553946 6259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:42223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:38:09.556061 6259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:09.557866 6259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:09.575085 6265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:09.575145 6269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:09.575165 6264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:09.576020 6259 server_base.cc:1034] running on GCE node
I20260806 22:38:09.576889 6259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:09.578151 6259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:09.579423 6259 hybrid_clock.cc:648] HybridClock initialized: now 1786055889579342 us; error 72 us; skew 500 ppm
I20260806 22:38:09.579841 6259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:38:09 dist-test-slave-h3ft krb5kdc[6115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055889, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:38:09.584432 6259 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:38:09.586465 6259 webserver.cc:460] Webserver started at http://127.15.198.129:33005/ using document root <none> and password file <none>
I20260806 22:38:09.587505 6259 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:09.587675 6259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:09.588119 6259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:38:09.592314 6259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "9550c8e6ef8c4773bd0aa9b80355afc1"
format_stamp: "Formatted at 2026-08-06 22:38:09 on dist-test-slave-h3ft"
server_key: "1e8598c15a40939ad23b1f20a17be967"
server_key_iv: "e4a690c3ddaec41eeef9100e018668a7"
server_key_version: "encryptionkey@0"
I20260806 22:38:09.593369 6259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "9550c8e6ef8c4773bd0aa9b80355afc1"
format_stamp: "Formatted at 2026-08-06 22:38:09 on dist-test-slave-h3ft"
server_key: "1e8598c15a40939ad23b1f20a17be967"
server_key_iv: "e4a690c3ddaec41eeef9100e018668a7"
server_key_version: "encryptionkey@0"
I20260806 22:38:09.600693 6259 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 22:38:09.605773 6278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:09.607908 6259 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:38:09.608136 6259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "9550c8e6ef8c4773bd0aa9b80355afc1"
format_stamp: "Formatted at 2026-08-06 22:38:09 on dist-test-slave-h3ft"
server_key: "1e8598c15a40939ad23b1f20a17be967"
server_key_iv: "e4a690c3ddaec41eeef9100e018668a7"
server_key_version: "encryptionkey@0"
I20260806 22:38:09.608402 6259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:09.626273 6259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:09.632062 6259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:09.632452 6259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:09.634011 6259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:09.636011 6259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:38:09.636112 6259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:09.636284 6259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:09.636369 6259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:09.678323 6259 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:45045
I20260806 22:38:09.678349 6431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:45045 every 8 connection(s)
I20260806 22:38:09.680178 6259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:38:09.683452 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 6259
I20260806 22:38:09.683759 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:38:09.684870 16154 external_mini_cluster.cc:1428] Setting key 34afb2eb706ab9b0f811350a8b51c34d
Aug 06 22:38:09 dist-test-slave-h3ft krb5kdc[6115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055889, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:38:09.713136 6434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:09.684095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:58929 (local address 127.15.198.190:33495)
0806 22:38:09.684813 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:09.684829 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:09.686281 (+ 1452us) server_negotiation.cc:407] Connection header received
0806 22:38:09.689403 (+ 3122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:09.689409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:09.689506 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:09.689624 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:09.692831 (+ 3207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:09.693745 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:09.696928 (+ 3183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:09.697511 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:09.697702 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:09.704777 (+ 7075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:09.704785 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:09.704789 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:09.704856 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:09.708549 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:09.709783 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:09.709789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:09.709792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:09.709994 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:09.710613 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:09.710618 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:09.710620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:09.710991 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:09.711265 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:09.712276 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0806 22:38:09.712682 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":299,"threads_started":1}
I20260806 22:38:09.714627 6435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:09.684878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:58929)
0806 22:38:09.686011 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:09.686075 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:09.688928 (+ 2853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:09.690036 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:09.690054 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:09.691255 (+ 1201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:09.692482 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:09.692523 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:09.693984 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:09.693993 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:09.696645 (+ 2652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:09.696672 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:09.697895 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:09.697907 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:09.698409 (+ 502us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:09.699711 (+ 1302us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:09.699759 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:09.704271 (+ 4512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:38:09.708848 (+ 4577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:09.708862 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:09.708885 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:09.709572 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:09.710140 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:09.710146 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:09.710150 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:09.710448 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:09.711222 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:38:09.711234 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:38:09.711772 (+ 538us) client_negotiation.cc:769] Sending connection context
0806 22:38:09.712119 (+ 347us) client_negotiation.cc:240] Negotiation successful
0806 22:38:09.712971 (+ 852us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"thread_start_us":419,"threads_started":1}
I20260806 22:38:09.716729 6432 heartbeater.cc:347] Connected to a master server at 127.15.198.190:33495
I20260806 22:38:09.717211 6432 heartbeater.cc:464] Registering TS with master...
I20260806 22:38:09.718241 6432 heartbeater.cc:511] Master 127.15.198.190:33495 requested a full tablet report, sending...
I20260806 22:38:09.721295 6175 ts_manager.cc:194] Registered new tserver with Master: 9550c8e6ef8c4773bd0aa9b80355afc1 (127.15.198.129:45045)
I20260806 22:38:09.723285 6175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:58929
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
I20260806 22:38:09.777256 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33495
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:42223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:38:10.231216 6440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:10.231633 6440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:10.231752 6440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:10.240032 6440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:10.240191 6440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:10.240324 6440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:38:10.252374 6440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:42223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:38:10.254240 6440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:10.256111 6440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:10.273200 6447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:10.273118 6452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:10.273190 6448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:10.280126 6440 server_base.cc:1034] running on GCE node
I20260806 22:38:10.281162 6440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:10.282538 6440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:10.283805 6440 hybrid_clock.cc:648] HybridClock initialized: now 1786055890283693 us; error 97 us; skew 500 ppm
I20260806 22:38:10.284332 6440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:38:10 dist-test-slave-h3ft krb5kdc[6115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055890, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:38:10.288937 6440 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:38:10.291375 6440 webserver.cc:460] Webserver started at http://127.15.198.130:45409/ using document root <none> and password file <none>
I20260806 22:38:10.292400 6440 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:10.292598 6440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:10.293135 6440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:38:10.297325 6440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "a47243ada0194498925303cfa3d672a2"
format_stamp: "Formatted at 2026-08-06 22:38:10 on dist-test-slave-h3ft"
server_key: "a4ef785b9c7ae7dc05becfa2a84e69df"
server_key_iv: "d76f8f362b9a62cc146c20f2c10e0ed7"
server_key_version: "encryptionkey@0"
I20260806 22:38:10.298496 6440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "a47243ada0194498925303cfa3d672a2"
format_stamp: "Formatted at 2026-08-06 22:38:10 on dist-test-slave-h3ft"
server_key: "a4ef785b9c7ae7dc05becfa2a84e69df"
server_key_iv: "d76f8f362b9a62cc146c20f2c10e0ed7"
server_key_version: "encryptionkey@0"
I20260806 22:38:10.306200 6440 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 22:38:10.311543 6460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:10.313630 6440 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260806 22:38:10.313885 6440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "a47243ada0194498925303cfa3d672a2"
format_stamp: "Formatted at 2026-08-06 22:38:10 on dist-test-slave-h3ft"
server_key: "a4ef785b9c7ae7dc05becfa2a84e69df"
server_key_iv: "d76f8f362b9a62cc146c20f2c10e0ed7"
server_key_version: "encryptionkey@0"
I20260806 22:38:10.314275 6440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:10.332082 6440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:10.338004 6440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:10.338431 6440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:10.339819 6440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:10.341573 6440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:38:10.341692 6440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:10.341866 6440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:10.341959 6440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:10.384066 6440 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:45185
I20260806 22:38:10.384119 6610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:45185 every 8 connection(s)
I20260806 22:38:10.385880 6440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:38:10.393857 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 6440
I20260806 22:38:10.394153 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:38:10.394881 16154 external_mini_cluster.cc:1428] Setting key 8ec55271b650cdf62f94e588826443f5
Aug 06 22:38:10 dist-test-slave-h3ft krb5kdc[6115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055890, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:38:10.414536 6614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:10.389284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:36839 (local address 127.15.198.190:33495)
0806 22:38:10.389855 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:10.389861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:10.391032 (+ 1171us) server_negotiation.cc:407] Connection header received
0806 22:38:10.394242 (+ 3210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:10.394251 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:10.394384 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:10.394548 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:10.396989 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:10.398210 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:10.400376 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:10.400916 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:10.401087 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:10.406709 (+ 5622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:10.406716 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:10.406720 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:10.406773 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:10.410175 (+ 3402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:10.411312 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:10.411318 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:10.411321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:10.411503 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:10.412063 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:10.412073 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:10.412076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:10.412482 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:10.412701 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:10.413702 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0806 22:38:10.414041 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":287,"threads_started":1}
I20260806 22:38:10.416008 6615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:10.389958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:36839)
0806 22:38:10.390801 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:10.390856 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:10.393078 (+ 2222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:10.394923 (+ 1845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:10.394938 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:10.395890 (+ 952us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:10.396765 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:10.396788 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:10.398405 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:10.398410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:10.400184 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:10.400200 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:10.401215 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:10.401225 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:10.401539 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:10.402494 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:10.402525 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:10.406258 (+ 3733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:38:10.410425 (+ 4167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:10.410441 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:10.410470 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:10.411131 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:10.411648 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:10.411654 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:10.411658 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:10.411918 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:10.412657 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:38:10.412668 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:38:10.413173 (+ 505us) client_negotiation.cc:769] Sending connection context
0806 22:38:10.413590 (+ 417us) client_negotiation.cc:240] Negotiation successful
0806 22:38:10.414371 (+ 781us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":314,"threads_started":1}
I20260806 22:38:10.418116 6612 heartbeater.cc:347] Connected to a master server at 127.15.198.190:33495
I20260806 22:38:10.418619 6612 heartbeater.cc:464] Registering TS with master...
I20260806 22:38:10.420001 6612 heartbeater.cc:511] Master 127.15.198.190:33495 requested a full tablet report, sending...
I20260806 22:38:10.422401 6175 ts_manager.cc:194] Registered new tserver with Master: a47243ada0194498925303cfa3d672a2 (127.15.198.130:45185)
I20260806 22:38:10.423561 6175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:36839
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
I20260806 22:38:10.489084 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33495
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:42223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:38:10.727487 6432 heartbeater.cc:503] Master 127.15.198.190:33495 was elected leader, sending a full tablet report...
W20260806 22:38:10.932442 6618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:10.932884 6618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:10.933073 6618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:10.941458 6618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:10.941672 6618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:10.941891 6618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:38:10.954140 6618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:42223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:38:10.956223 6618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:10.958033 6618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:10.975750 6624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:10.975718 6625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:10.976466 6628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:10.976747 6618 server_base.cc:1034] running on GCE node
I20260806 22:38:10.977591 6618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:10.978927 6618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:10.980232 6618 hybrid_clock.cc:648] HybridClock initialized: now 1786055890980178 us; error 42 us; skew 500 ppm
I20260806 22:38:10.980721 6618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:38:10 dist-test-slave-h3ft krb5kdc[6115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055890, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:38:10.990238 6618 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:38:10.992535 6618 webserver.cc:460] Webserver started at http://127.15.198.131:34829/ using document root <none> and password file <none>
I20260806 22:38:10.993535 6618 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:10.993667 6618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:10.994033 6618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:38:10.998376 6618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "9be77cfb3ddc4803a453828afbf575e0"
format_stamp: "Formatted at 2026-08-06 22:38:10 on dist-test-slave-h3ft"
server_key: "0a11f693e298c103cc8c88ebe9517c0b"
server_key_iv: "696e496a060592b31cab323880fca91a"
server_key_version: "encryptionkey@0"
I20260806 22:38:10.999442 6618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "9be77cfb3ddc4803a453828afbf575e0"
format_stamp: "Formatted at 2026-08-06 22:38:10 on dist-test-slave-h3ft"
server_key: "0a11f693e298c103cc8c88ebe9517c0b"
server_key_iv: "696e496a060592b31cab323880fca91a"
server_key_version: "encryptionkey@0"
I20260806 22:38:11.006383 6618 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260806 22:38:11.010964 6636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:11.012869 6618 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260806 22:38:11.013087 6618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "9be77cfb3ddc4803a453828afbf575e0"
format_stamp: "Formatted at 2026-08-06 22:38:10 on dist-test-slave-h3ft"
server_key: "0a11f693e298c103cc8c88ebe9517c0b"
server_key_iv: "696e496a060592b31cab323880fca91a"
server_key_version: "encryptionkey@0"
I20260806 22:38:11.013582 6618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:11.050318 6618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:11.056429 6618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:11.056834 6618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:11.058110 6618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:11.059813 6618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:38:11.059937 6618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:11.060123 6618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:11.060201 6618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:11.100186 6618 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:35351
I20260806 22:38:11.100236 6784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:35351 every 8 connection(s)
I20260806 22:38:11.101966 6618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:38:11.105002 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 6618
I20260806 22:38:11.105269 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:38:11.105896 16154 external_mini_cluster.cc:1428] Setting key 203bdcb9c8b2eb29e6a6a2c1c37b5621
Aug 06 22:38:11 dist-test-slave-h3ft krb5kdc[6115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055890, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:38:11.128670 6788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.105973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:48527 (local address 127.15.198.190:33495)
0806 22:38:11.106652 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:11.106659 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:11.108473 (+ 1814us) server_negotiation.cc:407] Connection header received
0806 22:38:11.110815 (+ 2342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:11.110820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:11.110920 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:11.111201 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:11.113144 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.114173 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.116082 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.116542 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.116643 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.121118 (+ 4475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:11.121124 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:11.121127 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:11.121167 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:11.124881 (+ 3714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:11.125840 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:11.125844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:11.125847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:11.125991 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:11.126563 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:11.126567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:11.126569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:11.126901 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:11.127215 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:11.128088 (+ 873us) server_negotiation.cc:299] Negotiation successful
0806 22:38:11.128331 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":285,"threads_started":1}
I20260806 22:38:11.130187 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.106874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:48527)
0806 22:38:11.108062 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:11.108135 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:11.110509 (+ 2374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:11.111459 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:11.111489 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:11.112155 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:11.112947 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.112969 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.114373 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.114377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.115927 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.115940 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.116781 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.116786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.117056 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.117944 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:11.117979 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:11.120843 (+ 2864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:38:11.125063 (+ 4220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:11.125072 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:11.125092 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:11.125713 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:11.126138 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:11.126142 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:11.126144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:11.126412 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:11.127115 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:38:11.127123 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:38:11.127621 (+ 498us) client_negotiation.cc:769] Sending connection context
0806 22:38:11.127929 (+ 308us) client_negotiation.cc:240] Negotiation successful
0806 22:38:11.128492 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"thread_start_us":456,"threads_started":1}
I20260806 22:38:11.132066 6785 heartbeater.cc:347] Connected to a master server at 127.15.198.190:33495
I20260806 22:38:11.132439 6785 heartbeater.cc:464] Registering TS with master...
I20260806 22:38:11.133414 6785 heartbeater.cc:511] Master 127.15.198.190:33495 requested a full tablet report, sending...
I20260806 22:38:11.135499 6175 ts_manager.cc:194] Registered new tserver with Master: 9be77cfb3ddc4803a453828afbf575e0 (127.15.198.131:35351)
I20260806 22:38:11.136411 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:38:11.136794 6175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:48527
I20260806 22:38:11.157641 6788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.142332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59136 (local address 127.15.198.190:33495)
0806 22:38:11.142582 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:11.142588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:11.143129 (+ 541us) server_negotiation.cc:407] Connection header received
0806 22:38:11.143337 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:11.143341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:11.143420 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:11.143518 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:11.145224 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.146076 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.147705 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.148198 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.148308 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.151151 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:11.151156 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:11.151160 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:11.151205 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:11.154383 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:11.155359 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:11.155364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:11.155366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:11.155539 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:11.155939 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:11.155946 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:11.155948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:11.156396 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:11.156622 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:11.156922 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 22:38:11.157295 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 22:38:11.162817 6175 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:38:11.171058 6175 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:38:11.201218 6802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.187558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:45045 (local address 127.0.0.1:34608)
0806 22:38:11.188722 (+ 1164us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:11.188780 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:11.189048 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:11.192034 (+ 2986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:11.192038 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:11.192090 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:38:11.192729 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.192742 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.194661 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.194668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.197691 (+ 3023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.197714 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.199948 (+ 2234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.199956 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.200264 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.200299 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:38:11.200389 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 22:38:11.200647 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"mutex_wait_us":154,"thread_start_us":1851,"threads_started":1}
I20260806 22:38:11.201874 6803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.187873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34608 (local address 127.15.198.129:45045)
0806 22:38:11.191394 (+ 3521us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:11.191403 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:11.191432 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:38:11.191538 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:11.191545 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:11.191785 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:11.192935 (+ 1150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:38:11.193173 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.194515 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.197890 (+ 3375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.199689 (+ 1799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.200594 (+ 905us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.200766 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:11.200856 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 22:38:11.201061 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3326,"thread_start_us":3132,"threads_started":1}
I20260806 22:38:11.201961 6805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.188434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42936 (local address 127.15.198.130:45185)
0806 22:38:11.190961 (+ 2527us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:11.190974 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:11.191037 (+ 63us) server_negotiation.cc:407] Connection header received
0806 22:38:11.191192 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:11.191203 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:11.191566 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:11.191808 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:38:11.193288 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.195072 (+ 1784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.198372 (+ 3300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.199990 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.200215 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.200399 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:11.201378 (+ 979us) server_negotiation.cc:299] Negotiation successful
0806 22:38:11.201563 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2297,"thread_start_us":2130,"threads_started":1}
I20260806 22:38:11.201272 6801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.187701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:45185 (local address 127.0.0.1:42936)
0806 22:38:11.190691 (+ 2990us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:11.190712 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:11.190831 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:11.191828 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:11.191837 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:11.191874 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:38:11.192513 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.192535 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.195234 (+ 2699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.195240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.198185 (+ 2945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.198199 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.200264 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.200272 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.200459 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.200521 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 22:38:11.200621 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 22:38:11.200758 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1988,"mutex_wait_us":428,"thread_start_us":455,"threads_started":1}
I20260806 22:38:11.202847 6806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.189389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:35351 (local address 127.0.0.1:42204)
0806 22:38:11.190542 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:11.190565 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:11.190706 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:11.192852 (+ 2146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:11.192857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:11.192883 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:38:11.193337 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.193350 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.195423 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.195437 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.197930 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.197946 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.199784 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.199790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.200075 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.200154 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 22:38:11.200332 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 22:38:11.202481 (+ 2149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1007,"mutex_wait_us":46,"thread_start_us":851,"threads_started":1}
I20260806 22:38:11.203240 6807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.190288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42204 (local address 127.15.198.131:35351)
0806 22:38:11.191852 (+ 1564us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:11.191860 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:11.191892 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:38:11.192007 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:11.192016 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:11.192217 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:11.192448 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:38:11.193596 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.195281 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.198106 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.199568 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.199802 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.200059 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:11.201739 (+ 1680us) server_negotiation.cc:299] Negotiation successful
0806 22:38:11.201939 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1390,"thread_start_us":1190,"threads_started":1}
I20260806 22:38:11.206879 6340 tablet_service.cc:1467] Processing CreateTablet for tablet 086657081d20441d924e0b58338237b2 (DEFAULT_TABLE table=test-table [id=7740b7b574c346828ed21ad85b20782c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:38:11.206885 6529 tablet_service.cc:1467] Processing CreateTablet for tablet 086657081d20441d924e0b58338237b2 (DEFAULT_TABLE table=test-table [id=7740b7b574c346828ed21ad85b20782c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:38:11.207222 6694 tablet_service.cc:1467] Processing CreateTablet for tablet 086657081d20441d924e0b58338237b2 (DEFAULT_TABLE table=test-table [id=7740b7b574c346828ed21ad85b20782c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:38:11.208772 6340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 086657081d20441d924e0b58338237b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:38:11.208799 6529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 086657081d20441d924e0b58338237b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:38:11.208789 6694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 086657081d20441d924e0b58338237b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:38:11.224610 6808 tablet_bootstrap.cc:492] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1: Bootstrap starting.
I20260806 22:38:11.229511 6808 tablet_bootstrap.cc:654] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1: Neither blocks nor log segments found. Creating new log.
I20260806 22:38:11.230710 6809 tablet_bootstrap.cc:492] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2: Bootstrap starting.
I20260806 22:38:11.231846 6810 tablet_bootstrap.cc:492] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0: Bootstrap starting.
I20260806 22:38:11.234266 6808 log.cc:824] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:38:11.236434 6810 tablet_bootstrap.cc:654] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0: Neither blocks nor log segments found. Creating new log.
I20260806 22:38:11.238072 6810 log.cc:824] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:38:11.238667 6809 tablet_bootstrap.cc:654] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2: Neither blocks nor log segments found. Creating new log.
I20260806 22:38:11.239077 6808 tablet_bootstrap.cc:492] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1: No bootstrap required, opened a new log
I20260806 22:38:11.239451 6808 ts_tablet_manager.cc:1397] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.006s
I20260806 22:38:11.240235 6809 log.cc:824] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:38:11.242153 6810 tablet_bootstrap.cc:492] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0: No bootstrap required, opened a new log
I20260806 22:38:11.242843 6810 ts_tablet_manager.cc:1397] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260806 22:38:11.243563 6809 tablet_bootstrap.cc:492] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2: No bootstrap required, opened a new log
I20260806 22:38:11.243882 6809 ts_tablet_manager.cc:1397] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.005s
I20260806 22:38:11.251730 6809 raft_consensus.cc:348] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
I20260806 22:38:11.252305 6809 raft_consensus.cc:374] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:38:11.252432 6809 raft_consensus.cc:729] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a47243ada0194498925303cfa3d672a2, State: Initialized, Role: FOLLOWER
I20260806 22:38:11.252185 6808 raft_consensus.cc:348] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
I20260806 22:38:11.252705 6808 raft_consensus.cc:374] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:38:11.252820 6808 raft_consensus.cc:729] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9550c8e6ef8c4773bd0aa9b80355afc1, State: Initialized, Role: FOLLOWER
I20260806 22:38:11.253511 6809 consensus_queue.cc:260] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
I20260806 22:38:11.253527 6808 consensus_queue.cc:260] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
I20260806 22:38:11.254773 6810 raft_consensus.cc:348] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
I20260806 22:38:11.255314 6810 raft_consensus.cc:374] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:38:11.255426 6810 raft_consensus.cc:729] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9be77cfb3ddc4803a453828afbf575e0, State: Initialized, Role: FOLLOWER
I20260806 22:38:11.256250 6808 ts_tablet_manager.cc:1428] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1: Time spent starting tablet: real 0.017s user 0.009s sys 0.007s
I20260806 22:38:11.256151 6810 consensus_queue.cc:260] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
I20260806 22:38:11.257529 6612 heartbeater.cc:503] Master 127.15.198.190:33495 was elected leader, sending a full tablet report...
I20260806 22:38:11.258116 6809 ts_tablet_manager.cc:1428] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260806 22:38:11.259727 6785 heartbeater.cc:503] Master 127.15.198.190:33495 was elected leader, sending a full tablet report...
I20260806 22:38:11.260394 6810 ts_tablet_manager.cc:1428] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0: Time spent starting tablet: real 0.017s user 0.017s sys 0.000s
I20260806 22:38:11.264470 6819 raft_consensus.cc:482] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:38:11.264820 6819 raft_consensus.cc:504] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
I20260806 22:38:11.267169 6819 leader_election.cc:288] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9550c8e6ef8c4773bd0aa9b80355afc1 (127.15.198.129:45045), a47243ada0194498925303cfa3d672a2 (127.15.198.130:45185)
I20260806 22:38:11.273918 6805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.268053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:59945 (local address 127.15.198.130:45185)
0806 22:38:11.268275 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:11.268281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:11.268300 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:38:11.268376 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:11.268382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:11.268474 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:11.268587 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:38:11.269264 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.270081 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.271990 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.272859 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.273055 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.273153 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:11.273444 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 22:38:11.273594 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 22:38:11.273918 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.267453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:45185 (local address 127.15.198.131:59945)
0806 22:38:11.267701 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:11.267725 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:11.267861 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:11.268791 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:11.268798 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:11.268830 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:38:11.269107 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.269117 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.270219 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.270227 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.271822 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.271838 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.273040 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.273046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.273251 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.273279 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 22:38:11.273414 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 22:38:11.273590 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":38}
I20260806 22:38:11.274242 6803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.267570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:39937 (local address 127.15.198.129:45045)
0806 22:38:11.268189 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:11.268196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:11.268522 (+ 326us) server_negotiation.cc:407] Connection header received
0806 22:38:11.268657 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:11.268662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:11.268748 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:11.268843 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:38:11.269573 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.270342 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.272241 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.273158 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.273362 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.273468 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:11.273778 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 22:38:11.273939 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440}
I20260806 22:38:11.275422 6821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.267656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:45045 (local address 127.15.198.131:39937)
0806 22:38:11.268430 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:11.268451 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:11.268555 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:11.268861 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:11.268865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:11.268900 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:38:11.269400 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.269420 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.270512 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.270517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.272061 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.272074 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.273390 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.273396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.273577 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.273614 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 22:38:11.274510 (+ 896us) client_negotiation.cc:240] Negotiation successful
0806 22:38:11.274668 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":25,"thread_start_us":583,"threads_started":1}
I20260806 22:38:11.275740 6553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "086657081d20441d924e0b58338237b2" candidate_uuid: "9be77cfb3ddc4803a453828afbf575e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a47243ada0194498925303cfa3d672a2" is_pre_election: true
I20260806 22:38:11.276444 6553 raft_consensus.cc:2454] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9be77cfb3ddc4803a453828afbf575e0 in term 0.
I20260806 22:38:11.276630 6364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "086657081d20441d924e0b58338237b2" candidate_uuid: "9be77cfb3ddc4803a453828afbf575e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" is_pre_election: true
I20260806 22:38:11.277282 6364 raft_consensus.cc:2454] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9be77cfb3ddc4803a453828afbf575e0 in term 0.
I20260806 22:38:11.277261 6637 leader_election.cc:302] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9be77cfb3ddc4803a453828afbf575e0, a47243ada0194498925303cfa3d672a2; no voters:
I20260806 22:38:11.277837 6819 raft_consensus.cc:2781] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:38:11.278035 6819 raft_consensus.cc:482] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:38:11.278170 6819 raft_consensus.cc:3037] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:38:11.280967 6819 raft_consensus.cc:504] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
I20260806 22:38:11.281730 6819 leader_election.cc:288] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [CANDIDATE]: Term 1 election: Requested vote from peers 9550c8e6ef8c4773bd0aa9b80355afc1 (127.15.198.129:45045), a47243ada0194498925303cfa3d672a2 (127.15.198.130:45185)
I20260806 22:38:11.282392 6364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "086657081d20441d924e0b58338237b2" candidate_uuid: "9be77cfb3ddc4803a453828afbf575e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1"
I20260806 22:38:11.282395 6553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "086657081d20441d924e0b58338237b2" candidate_uuid: "9be77cfb3ddc4803a453828afbf575e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a47243ada0194498925303cfa3d672a2"
I20260806 22:38:11.282680 6553 raft_consensus.cc:3037] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:38:11.282657 6364 raft_consensus.cc:3037] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:38:11.285341 6364 raft_consensus.cc:2454] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9be77cfb3ddc4803a453828afbf575e0 in term 1.
I20260806 22:38:11.285334 6553 raft_consensus.cc:2454] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9be77cfb3ddc4803a453828afbf575e0 in term 1.
I20260806 22:38:11.285885 6637 leader_election.cc:302] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9be77cfb3ddc4803a453828afbf575e0, a47243ada0194498925303cfa3d672a2; no voters:
I20260806 22:38:11.286252 6819 raft_consensus.cc:2781] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:38:11.286819 6819 raft_consensus.cc:686] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [term 1 LEADER]: Becoming Leader. State: Replica: 9be77cfb3ddc4803a453828afbf575e0, State: Running, Role: LEADER
I20260806 22:38:11.287454 6819 consensus_queue.cc:237] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
I20260806 22:38:11.293432 6175 catalog_manager.cc:5495] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9be77cfb3ddc4803a453828afbf575e0 (127.15.198.131). New cstate: current_term: 1 leader_uuid: "9be77cfb3ddc4803a453828afbf575e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } health_report { overall_health: HEALTHY } } }
I20260806 22:38:11.335988 6807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.329005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42218 (local address 127.15.198.131:35351)
0806 22:38:11.329231 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:11.329237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:11.329258 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:38:11.329399 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:11.329404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:11.329545 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:11.329744 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:38:11.330520 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.331597 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.333243 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.333738 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.333860 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.334513 (+ 653us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:38:11.335381 (+ 868us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:38:11.335526 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:11.335615 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 22:38:11.335709 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 22:38:11.347623 6364 raft_consensus.cc:1261] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 1 FOLLOWER]: Refusing update from remote peer 9be77cfb3ddc4803a453828afbf575e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:38:11.347623 6553 raft_consensus.cc:1261] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2 [term 1 FOLLOWER]: Refusing update from remote peer 9be77cfb3ddc4803a453828afbf575e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:38:11.349138 6822 consensus_queue.cc:1035] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:38:11.349646 6819 consensus_queue.cc:1035] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:38:11.354064 6787 tablet.cc:2369] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:38:11.365792 6829 mvcc.cc:205] Tried to move back new op lower bound from 7315684930944086016 to 7315684930718404608. Current Snapshot: MvccSnapshot[applied={T|T < 7315684930944086016}]
I20260806 22:38:11.392210 6175 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:38:11.392632 6175 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:38:11.399127 6175 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Sending DeleteTablet for 3 replicas of tablet 086657081d20441d924e0b58338237b2
I20260806 22:38:11.400867 6340 tablet_service.cc:1514] Processing DeleteTablet for tablet 086657081d20441d924e0b58338237b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:38:11 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:34608
I20260806 22:38:11.401571 6694 tablet_service.cc:1514] Processing DeleteTablet for tablet 086657081d20441d924e0b58338237b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:38:11 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:42204
I20260806 22:38:11.402513 6529 tablet_service.cc:1514] Processing DeleteTablet for tablet 086657081d20441d924e0b58338237b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:38:11 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:42936
I20260806 22:38:11.402733 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 6259
I20260806 22:38:11.418221 6844 tablet_replica.cc:330] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1: stopping tablet replica
I20260806 22:38:11.419278 6844 raft_consensus.cc:2229] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:38:11.420313 6844 raft_consensus.cc:2258] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:38:11.420764 6848 tablet_replica.cc:330] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2: stopping tablet replica
I20260806 22:38:11.421376 6848 raft_consensus.cc:2229] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:38:11.422155 6848 raft_consensus.cc:2258] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:38:11.424544 6844 ts_tablet_manager.cc:1905] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:38:11.427317 6848 ts_tablet_manager.cc:1905] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 22:38:11 dist-test-slave-h3ft krb5kdc[6115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:38:11.428966 6846 tablet_replica.cc:330] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0: stopping tablet replica
I20260806 22:38:11.429625 6846 raft_consensus.cc:2229] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:38:11.433717 6844 ts_tablet_manager.cc:1918] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:38:11.433998 6844 log.cc:1196] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/086657081d20441d924e0b58338237b2
I20260806 22:38:11.434666 6844 ts_tablet_manager.cc:1939] T 086657081d20441d924e0b58338237b2 P 9550c8e6ef8c4773bd0aa9b80355afc1: Deleting consensus metadata
I20260806 22:38:11.436627 6848 ts_tablet_manager.cc:1918] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:38:11.436849 6848 log.cc:1196] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/086657081d20441d924e0b58338237b2
I20260806 22:38:11.437493 6848 ts_tablet_manager.cc:1939] T 086657081d20441d924e0b58338237b2 P a47243ada0194498925303cfa3d672a2: Deleting consensus metadata
I20260806 22:38:11.437460 6154 catalog_manager.cc:4841] TS 9550c8e6ef8c4773bd0aa9b80355afc1 (127.15.198.129:45045): tablet 086657081d20441d924e0b58338237b2 (table test-table [id=7740b7b574c346828ed21ad85b20782c]) successfully deleted
I20260806 22:38:11.440039 6153 catalog_manager.cc:4841] TS a47243ada0194498925303cfa3d672a2 (127.15.198.130:45185): tablet 086657081d20441d924e0b58338237b2 (table test-table [id=7740b7b574c346828ed21ad85b20782c]) successfully deleted
I20260806 22:38:11.441774 6803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.404079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34614 (local address 127.15.198.129:45045)
0806 22:38:11.404389 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:11.404431 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:11.411826 (+ 7395us) server_negotiation.cc:407] Connection header received
0806 22:38:11.412266 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:11.412277 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:11.412430 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:11.412538 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:11.415963 (+ 3425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.417709 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.421249 (+ 3540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.422034 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.422194 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.430942 (+ 8748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:11.430970 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:11.430995 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:11.431126 (+ 131us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:11.436289 (+ 5163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:11.438110 (+ 1821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:11.438122 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:11.438126 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:11.438341 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:11.438997 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:11.439031 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:11.439034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:11.439633 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:11.439973 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:11.441062 (+ 1089us) server_negotiation.cc:299] Negotiation successful
0806 22:38:11.441343 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 22:38:11.441854 6846 raft_consensus.cc:2258] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:38:11.442836 6423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:38:11.445032 6846 ts_tablet_manager.cc:1905] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:38:11.451831 6846 ts_tablet_manager.cc:1918] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:38:11.452102 6846 log.cc:1196] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/086657081d20441d924e0b58338237b2
I20260806 22:38:11.452701 6846 ts_tablet_manager.cc:1939] T 086657081d20441d924e0b58338237b2 P 9be77cfb3ddc4803a453828afbf575e0: Deleting consensus metadata
I20260806 22:38:11.455139 6153 catalog_manager.cc:4841] TS 9be77cfb3ddc4803a453828afbf575e0 (127.15.198.131:35351): tablet 086657081d20441d924e0b58338237b2 (table test-table [id=7740b7b574c346828ed21ad85b20782c]) successfully deleted
I20260806 22:38:11.563859 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 6259
I20260806 22:38:11.606297 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 6440
Aug 06 22:38:11 dist-test-slave-h3ft krb5kdc[6115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.130@KRBTEST.COM
I20260806 22:38:11.633795 6805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.607655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42952 (local address 127.15.198.130:45185)
0806 22:38:11.607901 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:11.607908 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:11.607933 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:38:11.608146 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:11.608153 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:11.608259 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:11.608408 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:11.610091 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.611362 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.612958 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.613640 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.613775 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.624271 (+ 10496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:11.624307 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:11.624336 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:11.624414 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:11.629275 (+ 4861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:11.630406 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:11.630419 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:11.630423 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:11.630641 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:11.631349 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:11.631355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:11.631359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:11.632219 (+ 860us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:11.632715 (+ 496us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:11.633020 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 22:38:11.633344 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 22:38:11.634814 6604 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:38:11.736764 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 6440
I20260806 22:38:11.768941 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 6618
Aug 06 22:38:11 dist-test-slave-h3ft krb5kdc[6115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.131@KRBTEST.COM
I20260806 22:38:11.793002 6807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.770238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42230 (local address 127.15.198.131:35351)
0806 22:38:11.770455 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:11.770461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:11.770487 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:38:11.770567 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:11.770572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:11.770720 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:11.770867 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:11.773139 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.774180 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.776693 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.777269 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.777457 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.784534 (+ 7077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:11.784559 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:11.784583 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:11.784655 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:11.788684 (+ 4029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:11.789988 (+ 1304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:11.789998 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:11.790001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:11.790237 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:11.790840 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:11.790845 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:11.790847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:11.791534 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:11.791957 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:11.792340 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 22:38:11.792587 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:38:11.794015 6776 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:38:11.944849 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 6618
I20260806 22:38:11.982805 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 6129
I20260806 22:38:11.984467 6215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:38:12.117800 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 6129
I20260806 22:38:32.154372 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:33495
--webserver_interface=127.15.198.190
--webserver_port=38655
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:42223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:33495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:38:32.629300 6857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:32.629803 6857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:32.630010 6857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:32.638613 6857 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:38:32.638792 6857 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:38:32.638916 6857 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:38:32.639043 6857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:38:32.639160 6857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:32.639298 6857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:32.639396 6857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:38:32.639485 6857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:38:32.651320 6857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:42223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:33495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:33495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=38655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:38:32.653327 6857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:32.655642 6857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:32.690786 6864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:32.690697 6867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:32.690754 6865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:32.691864 6857 server_base.cc:1034] running on GCE node
I20260806 22:38:32.693152 6857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:32.695557 6857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:32.696877 6857 hybrid_clock.cc:648] HybridClock initialized: now 1786055912696757 us; error 81 us; skew 500 ppm
I20260806 22:38:32.697463 6857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:38:32 dist-test-slave-h3ft krb5kdc[6115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055912, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:38:32.707782 6857 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:38:32.710444 6857 webserver.cc:460] Webserver started at http://127.15.198.190:38655/ using document root <none> and password file <none>
I20260806 22:38:32.711689 6857 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:32.711836 6857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:32.720042 6857 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 22:38:32.724383 6875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:32.726356 6857 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:38:32.726570 6857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "c042426a636e498b9996a841635e7dff"
format_stamp: "Formatted at 2026-08-06 22:38:08 on dist-test-slave-h3ft"
server_key: "a548424a82b091aa7dba34c3524f8931"
server_key_iv: "1a1eb6d0b5b3265a545e1c8daa1b7d68"
server_key_version: "encryptionkey@0"
I20260806 22:38:32.728053 6857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:32.744793 6857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:32.751060 6857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:32.751401 6857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:32.776157 6857 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:33495
I20260806 22:38:32.776186 6940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:33495 every 8 connection(s)
I20260806 22:38:32.778512 6857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:38:32.782183 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 6857
I20260806 22:38:32.783560 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:45045
--local_ip_for_outbound_sockets=127.15.198.129
--tserver_master_addrs=127.15.198.190:33495
--webserver_port=33005
--webserver_interface=127.15.198.129
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:42223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:38:32.792294 6941 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:38:32.799773 6941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Bootstrap starting.
I20260806 22:38:32.829087 6941 log.cc:824] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Log is configured to *not* fsync() on all Append() calls
I20260806 22:38:32.848165 6941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:38:32.849131 6941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Bootstrap complete.
I20260806 22:38:32.862898 6941 raft_consensus.cc:348] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } }
I20260806 22:38:32.864190 6941 raft_consensus.cc:729] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c042426a636e498b9996a841635e7dff, State: Initialized, Role: FOLLOWER
I20260806 22:38:32.865072 6941 consensus_queue.cc:260] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } }
I20260806 22:38:32.865367 6941 raft_consensus.cc:388] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:38:32.865513 6941 raft_consensus.cc:482] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:38:32.865734 6941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:38:32.871068 6941 raft_consensus.cc:504] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } }
I20260806 22:38:32.871868 6941 leader_election.cc:302] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c042426a636e498b9996a841635e7dff; no voters:
I20260806 22:38:32.872776 6941 leader_election.cc:288] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:38:32.872984 6946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:38:32.874322 6946 raft_consensus.cc:686] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 2 LEADER]: Becoming Leader. State: Replica: c042426a636e498b9996a841635e7dff, State: Running, Role: LEADER
I20260806 22:38:32.875051 6946 consensus_queue.cc:237] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } }
I20260806 22:38:32.875859 6941 sys_catalog.cc:564] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:38:32.878279 6948 sys_catalog.cc:455] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [sys.catalog]: SysCatalogTable state changed. Reason: New leader c042426a636e498b9996a841635e7dff. Latest consensus state: current_term: 2 leader_uuid: "c042426a636e498b9996a841635e7dff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } } }
I20260806 22:38:32.878821 6948 sys_catalog.cc:458] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [sys.catalog]: This master's current role is: LEADER
I20260806 22:38:32.878631 6947 sys_catalog.cc:455] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c042426a636e498b9996a841635e7dff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } } }
I20260806 22:38:32.879215 6947 sys_catalog.cc:458] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [sys.catalog]: This master's current role is: LEADER
I20260806 22:38:32.884264 6952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:38:32.894210 6952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:38:32.896240 6952 catalog_manager.cc:1259] Loaded cluster ID: 335287833ba049098ee01aa86dbdfa52
I20260806 22:38:32.896394 6952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:38:32.901058 6952 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:38:32.903451 6952 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Loaded TSK: 0
I20260806 22:38:32.950124 6952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Generated new TSK 1
I20260806 22:38:32.950875 6952 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:38:33.320192 6943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:33.320677 6943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:33.320895 6943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:33.329655 6943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:33.329833 6943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:33.330049 6943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:38:33.342424 6943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:42223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:45045
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=33005
--tserver_master_addrs=127.15.198.190:33495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:38:33.344307 6943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:33.346037 6943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:33.362818 6977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:33.362865 6981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:33.362794 6978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:33.363955 6943 server_base.cc:1034] running on GCE node
I20260806 22:38:33.364800 6943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:33.366138 6943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:33.367422 6943 hybrid_clock.cc:648] HybridClock initialized: now 1786055913367338 us; error 71 us; skew 500 ppm
I20260806 22:38:33.367857 6943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:38:33 dist-test-slave-h3ft krb5kdc[6115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055913, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:38:33.372002 6943 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:38:33.374208 6943 webserver.cc:460] Webserver started at http://127.15.198.129:33005/ using document root <none> and password file <none>
I20260806 22:38:33.375232 6943 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:33.375396 6943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:33.382774 6943 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:38:33.386570 6989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:33.388665 6943 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:38:33.388876 6943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "9550c8e6ef8c4773bd0aa9b80355afc1"
format_stamp: "Formatted at 2026-08-06 22:38:09 on dist-test-slave-h3ft"
server_key: "1e8598c15a40939ad23b1f20a17be967"
server_key_iv: "e4a690c3ddaec41eeef9100e018668a7"
server_key_version: "encryptionkey@0"
I20260806 22:38:33.390156 6943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:33.415975 6943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:33.422150 6943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:33.422458 6943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:33.423780 6943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:33.425532 6943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:38:33.425632 6943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:33.425726 6943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:33.425786 6943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:33.467127 6943 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:45045
I20260806 22:38:33.467197 7146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:45045 every 8 connection(s)
I20260806 22:38:33.469092 6943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:38:33.475311 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 6943
I20260806 22:38:33.476154 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:45185
--local_ip_for_outbound_sockets=127.15.198.130
--tserver_master_addrs=127.15.198.190:33495
--webserver_port=45409
--webserver_interface=127.15.198.130
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:42223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:38:33 dist-test-slave-h3ft krb5kdc[6115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055913, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:38:33.502249 7150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:33.472833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:41203 (local address 127.15.198.190:33495)
0806 22:38:33.473583 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:33.473598 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:33.474419 (+ 821us) server_negotiation.cc:407] Connection header received
0806 22:38:33.477453 (+ 3034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:33.477487 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:33.478114 (+ 627us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:33.478659 (+ 545us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:33.480555 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:33.482670 (+ 2115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:33.485529 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:33.486721 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:33.487052 (+ 331us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:33.491547 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:33.491558 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:33.491579 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:33.491632 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:33.496347 (+ 4715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:33.497368 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:33.497377 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:33.497383 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:33.497604 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:33.498151 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:33.498163 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:33.498166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:33.498807 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:33.499103 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:33.500060 (+ 957us) server_negotiation.cc:299] Negotiation successful
0806 22:38:33.500578 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":334,"threads_started":1}
I20260806 22:38:33.502303 7152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:33.473169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:41203)
0806 22:38:33.474132 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:33.474202 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:33.476591 (+ 2389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:33.478841 (+ 2250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:33.478854 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:33.479601 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:33.480276 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:33.480297 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:33.482926 (+ 2629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:33.482934 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:33.485308 (+ 2374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:33.485333 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:33.487115 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:33.487125 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:33.487482 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:33.488271 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:33.488307 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:33.491151 (+ 2844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:38:33.496572 (+ 5421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:33.496590 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:33.496608 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:33.497165 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:33.497746 (+ 581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:33.497751 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:33.497755 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:33.498007 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:33.498974 (+ 967us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:38:33.498985 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:38:33.499465 (+ 480us) client_negotiation.cc:769] Sending connection context
0806 22:38:33.499798 (+ 333us) client_negotiation.cc:240] Negotiation successful
0806 22:38:33.500578 (+ 780us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":402,"threads_started":1}
I20260806 22:38:33.506448 7147 heartbeater.cc:347] Connected to a master server at 127.15.198.190:33495
I20260806 22:38:33.506903 7147 heartbeater.cc:464] Registering TS with master...
I20260806 22:38:33.508167 7147 heartbeater.cc:511] Master 127.15.198.190:33495 requested a full tablet report, sending...
I20260806 22:38:33.511709 6896 ts_manager.cc:194] Registered new tserver with Master: 9550c8e6ef8c4773bd0aa9b80355afc1 (127.15.198.129:45045)
I20260806 22:38:33.514591 6896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:41203
W20260806 22:38:33.933275 7153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:33.933710 7153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:33.933833 7153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:33.942520 7153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:33.942703 7153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:33.942839 7153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:38:33.954792 7153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:42223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:45185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=45409
--tserver_master_addrs=127.15.198.190:33495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:38:33.956854 7153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:33.958694 7153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:33.976761 7160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:33.976898 7162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:33.976752 7159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:33.978076 7153 server_base.cc:1034] running on GCE node
I20260806 22:38:33.979141 7153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:33.980497 7153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:33.981823 7153 hybrid_clock.cc:648] HybridClock initialized: now 1786055913981746 us; error 102 us; skew 500 ppm
I20260806 22:38:33.982285 7153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:38:33 dist-test-slave-h3ft krb5kdc[6115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055913, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:38:33.986464 7153 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:38:33.988605 7153 webserver.cc:460] Webserver started at http://127.15.198.130:45409/ using document root <none> and password file <none>
I20260806 22:38:33.989717 7153 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:33.990160 7153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:33.997874 7153 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 22:38:34.001714 7169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:34.003952 7153 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:38:34.004209 7153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "a47243ada0194498925303cfa3d672a2"
format_stamp: "Formatted at 2026-08-06 22:38:10 on dist-test-slave-h3ft"
server_key: "a4ef785b9c7ae7dc05becfa2a84e69df"
server_key_iv: "d76f8f362b9a62cc146c20f2c10e0ed7"
server_key_version: "encryptionkey@0"
I20260806 22:38:34.005494 7153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:34.035864 7153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:34.041761 7153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:34.042126 7153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:34.043481 7153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:34.045296 7153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:38:34.045441 7153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:34.045593 7153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:34.045692 7153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:34.086865 7153 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:45185
I20260806 22:38:34.086916 7315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:45185 every 8 connection(s)
I20260806 22:38:34.088941 7153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:38:34.097240 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 7153
I20260806 22:38:34.098125 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:35351
--local_ip_for_outbound_sockets=127.15.198.131
--tserver_master_addrs=127.15.198.190:33495
--webserver_port=34829
--webserver_interface=127.15.198.131
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:42223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:38:34 dist-test-slave-h3ft krb5kdc[6115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055913, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:38:34.115374 7318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:34.092526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:48187 (local address 127.15.198.190:33495)
0806 22:38:34.093222 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:34.093227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:34.094170 (+ 943us) server_negotiation.cc:407] Connection header received
0806 22:38:34.096723 (+ 2553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:34.096730 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:34.096850 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:34.097003 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:34.099410 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.100241 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.102135 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.102633 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.102803 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:34.107466 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:34.107473 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:34.107478 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:34.107535 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:34.111100 (+ 3565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:34.112196 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:34.112201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:34.112204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:34.112373 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:34.112935 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:34.112939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:34.112942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:34.113436 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:34.113622 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:34.114628 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0806 22:38:34.114966 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":354,"threads_started":1}
I20260806 22:38:34.116691 7320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:34.093061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:48187)
0806 22:38:34.093939 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:34.094002 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:34.096318 (+ 2316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:34.097333 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:34.097346 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:34.098213 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:34.099167 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:34.099194 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.100462 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.100467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:34.101944 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:34.101960 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.102939 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.102949 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:34.103313 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:34.104239 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:34.104269 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:34.107130 (+ 2861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:38:34.111357 (+ 4227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:34.111370 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:34.111400 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:34.112020 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:34.112514 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:34.112519 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:34.112523 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:34.112788 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:34.113629 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:38:34.113637 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:38:34.114145 (+ 508us) client_negotiation.cc:769] Sending connection context
0806 22:38:34.114454 (+ 309us) client_negotiation.cc:240] Negotiation successful
0806 22:38:34.115101 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":336,"threads_started":1}
I20260806 22:38:34.118749 7316 heartbeater.cc:347] Connected to a master server at 127.15.198.190:33495
I20260806 22:38:34.119239 7316 heartbeater.cc:464] Registering TS with master...
I20260806 22:38:34.120335 7316 heartbeater.cc:511] Master 127.15.198.190:33495 requested a full tablet report, sending...
I20260806 22:38:34.122354 6896 ts_manager.cc:194] Registered new tserver with Master: a47243ada0194498925303cfa3d672a2 (127.15.198.130:45185)
I20260806 22:38:34.123452 6896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:48187
I20260806 22:38:34.520731 7147 heartbeater.cc:503] Master 127.15.198.190:33495 was elected leader, sending a full tablet report...
W20260806 22:38:34.546778 7321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:34.547326 7321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:34.547528 7321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:34.555966 7321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:34.556192 7321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:34.556406 7321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:38:34.568240 7321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:42223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:35351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=34829
--tserver_master_addrs=127.15.198.190:33495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:38:34.570179 7321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:34.572175 7321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:34.589780 7329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:34.589768 7330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:34.589820 7332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:34.590833 7321 server_base.cc:1034] running on GCE node
I20260806 22:38:34.591763 7321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:34.593106 7321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:34.594405 7321 hybrid_clock.cc:648] HybridClock initialized: now 1786055914594313 us; error 72 us; skew 500 ppm
I20260806 22:38:34.594938 7321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:38:34 dist-test-slave-h3ft krb5kdc[6115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055914, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:38:34.599458 7321 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:38:34.601553 7321 webserver.cc:460] Webserver started at http://127.15.198.131:34829/ using document root <none> and password file <none>
I20260806 22:38:34.602567 7321 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:34.602772 7321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:34.610006 7321 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 22:38:34.613873 7341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:34.616003 7321 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 22:38:34.616247 7321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "9be77cfb3ddc4803a453828afbf575e0"
format_stamp: "Formatted at 2026-08-06 22:38:10 on dist-test-slave-h3ft"
server_key: "0a11f693e298c103cc8c88ebe9517c0b"
server_key_iv: "696e496a060592b31cab323880fca91a"
server_key_version: "encryptionkey@0"
I20260806 22:38:34.617589 7321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:34.644958 7321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:34.650580 7321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:34.650897 7321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:34.652256 7321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:34.654181 7321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:38:34.654300 7321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:34.654506 7321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:34.654596 7321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:34.696376 7321 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:35351
I20260806 22:38:34.696449 7491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:35351 every 8 connection(s)
I20260806 22:38:34.698448 7321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:38:34.704418 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 7321
Aug 06 22:38:34 dist-test-slave-h3ft krb5kdc[6115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055914, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:38:34.724113 7501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:34.705931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34156 (local address 127.15.198.190:33495)
0806 22:38:34.707090 (+ 1159us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:34.707095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:34.707113 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:38:34.707181 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:34.707185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:34.707269 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:34.707409 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:34.709411 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.710448 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.712406 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.712910 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.713067 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:34.716659 (+ 3592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:34.716665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:34.716669 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:34.716709 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:34.720427 (+ 3718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:34.721439 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:34.721448 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:34.721452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:34.721633 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:34.722164 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:34.722169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:34.722171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:34.722599 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:34.723062 (+ 463us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:34.723385 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 22:38:34.723716 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":964,"thread_start_us":755,"threads_started":1}
I20260806 22:38:34.726377 7496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:34.702206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:53225 (local address 127.15.198.190:33495)
0806 22:38:34.702833 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:34.702838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:34.704538 (+ 1700us) server_negotiation.cc:407] Connection header received
0806 22:38:34.707749 (+ 3211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:34.707754 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:34.707854 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:34.708010 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:34.710151 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.711223 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.713159 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.713622 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.713767 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:34.718498 (+ 4731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:34.718523 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:34.718528 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:34.718611 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:34.722005 (+ 3394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:34.723172 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:34.723181 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:34.723184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:34.723377 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:34.724078 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:34.724083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:34.724086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:34.724529 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:34.724738 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:34.725746 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0806 22:38:34.726035 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":309,"threads_started":1}
I20260806 22:38:34.727959 7498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:34.702925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:53225)
0806 22:38:34.704134 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:34.704200 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:34.707381 (+ 3181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:34.708396 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:34.708409 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:34.709109 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:34.709901 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:34.709923 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.711419 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.711423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:34.712996 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:34.713011 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.713913 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.713922 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:34.714221 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:34.715106 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:34.715139 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:34.718171 (+ 3032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:38:34.722214 (+ 4043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:34.722229 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:34.722257 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:34.722988 (+ 731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:34.723524 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:34.723529 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:34.723532 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:34.723908 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:34.724699 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:38:34.724710 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:38:34.725281 (+ 571us) client_negotiation.cc:769] Sending connection context
0806 22:38:34.725576 (+ 295us) client_negotiation.cc:240] Negotiation successful
0806 22:38:34.726317 (+ 741us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"thread_start_us":390,"threads_started":1}
I20260806 22:38:34.729665 7494 heartbeater.cc:347] Connected to a master server at 127.15.198.190:33495
I20260806 22:38:34.730084 7494 heartbeater.cc:464] Registering TS with master...
I20260806 22:38:34.731068 7494 heartbeater.cc:511] Master 127.15.198.190:33495 requested a full tablet report, sending...
I20260806 22:38:34.732936 6896 ts_manager.cc:194] Registered new tserver with Master: 9be77cfb3ddc4803a453828afbf575e0 (127.15.198.131:35351)
I20260806 22:38:34.733985 6896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:53225
I20260806 22:38:34.740935 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:38:34.749694 7496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:34.742205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34172 (local address 127.15.198.190:33495)
0806 22:38:34.742446 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:34.742451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:34.743182 (+ 731us) server_negotiation.cc:407] Connection header received
0806 22:38:34.743460 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:34.743465 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:34.743651 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:34.743821 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:38:34.744381 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.745274 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.746933 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.747683 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.747820 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:34.748506 (+ 686us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:38:34.748907 (+ 401us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 22:38:34.749362 (+ 455us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34172: token expired
Metrics: {"server-negotiator.queue_time_us":97}
W20260806 22:38:34.749902 7496 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34172: token expired
W20260806 22:38:34.750542 6791 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.198.190:33495)
I20260806 22:38:34.765815 7496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:34.751366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34180 (local address 127.15.198.190:33495)
0806 22:38:34.751553 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:34.751557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:34.751574 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:38:34.751737 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:34.751740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:34.751808 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:34.751890 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:34.753556 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.754349 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.755918 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.756373 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.756475 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:34.760059 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:34.760065 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:34.760069 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:34.760108 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:34.762876 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:34.763706 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:34.763710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:34.763712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:34.763861 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:34.764215 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:34.764218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:34.764220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:34.764595 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:34.764828 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:34.765195 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 22:38:34.765402 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:38:34.770377 6896 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:38:34.779897 6896 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:38:34.813081 7507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:34.797408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:45045 (local address 127.0.0.1:41566)
0806 22:38:34.798854 (+ 1446us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:34.798912 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:34.799166 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:34.802779 (+ 3613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:34.802788 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:34.802832 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:38:34.803494 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:34.803510 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.806112 (+ 2602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.806118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:34.808983 (+ 2865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:34.809001 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.811705 (+ 2704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.811712 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:34.811961 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:34.812039 (+ 78us) client_negotiation.cc:769] Sending connection context
0806 22:38:34.812190 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 22:38:34.812470 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1092,"thread_start_us":1314,"threads_started":1}
I20260806 22:38:34.814100 7509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:34.797738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41566 (local address 127.15.198.129:45045)
0806 22:38:34.801865 (+ 4127us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:34.801874 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:34.801916 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:38:34.802075 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:34.802086 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:34.802457 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:34.803769 (+ 1312us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:38:34.804081 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.805924 (+ 1843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.809255 (+ 3331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.811258 (+ 2003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.813135 (+ 1877us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:34.813359 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:34.813484 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 22:38:34.813718 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3864,"thread_start_us":3678,"threads_started":1}
I20260806 22:38:34.815109 7508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:34.797697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:45185 (local address 127.0.0.1:58430)
0806 22:38:34.801920 (+ 4223us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:34.801943 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:34.802104 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:34.803200 (+ 1096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:34.803207 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:34.803239 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:38:34.804658 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:34.804674 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.806778 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.806784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:34.810105 (+ 3321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:34.810125 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.812735 (+ 2610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.812741 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:34.814430 (+ 1689us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:34.814467 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 22:38:34.814553 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:38:34.814720 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4051,"thread_start_us":3872,"threads_started":1}
I20260806 22:38:34.815613 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:34.798498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58430 (local address 127.15.198.130:45185)
0806 22:38:34.801329 (+ 2831us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:34.801343 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:34.802250 (+ 907us) server_negotiation.cc:407] Connection header received
0806 22:38:34.802370 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:34.802377 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:34.802590 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:34.802806 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:38:34.804893 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.806589 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.810356 (+ 3767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.812108 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.812281 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:34.812495 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:34.814908 (+ 2413us) server_negotiation.cc:299] Negotiation successful
0806 22:38:34.815177 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2598,"thread_start_us":2407,"threads_started":1}
I20260806 22:38:34.820521 7237 tablet_service.cc:1467] Processing CreateTablet for tablet 64ed6a5ff98c4586bbee424444ef3d07 (DEFAULT_TABLE table=test-table [id=f07d00d19d594ae09e2de7a3dc98deef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:38:34.820753 7060 tablet_service.cc:1467] Processing CreateTablet for tablet 64ed6a5ff98c4586bbee424444ef3d07 (DEFAULT_TABLE table=test-table [id=f07d00d19d594ae09e2de7a3dc98deef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:38:34.822646 7237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64ed6a5ff98c4586bbee424444ef3d07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:38:34.822707 7060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64ed6a5ff98c4586bbee424444ef3d07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:38:34.825896 7512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:34.802587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46404 (local address 127.15.198.131:35351)
0806 22:38:34.816931 (+ 14344us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:34.816942 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:34.816994 (+ 52us) server_negotiation.cc:407] Connection header received
0806 22:38:34.817136 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:34.817146 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:34.817488 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:34.817818 (+ 330us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:38:34.818417 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.820232 (+ 1815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.822756 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.824470 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.824748 (+ 278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:34.824976 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:34.825276 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 22:38:34.825496 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14113,"thread_start_us":13943,"threads_started":1}
I20260806 22:38:34.826798 7507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:34.802165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:35351 (local address 127.0.0.1:46404)
0806 22:38:34.813450 (+ 11285us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:34.813477 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:34.813606 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:34.817812 (+ 4206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:34.817821 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:34.817870 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:38:34.818183 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:34.818200 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.820420 (+ 2220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.820426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:34.822454 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:34.822485 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.824731 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.824739 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:34.825003 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:34.825056 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 22:38:34.825197 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 22:38:34.825424 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11122,"thread_start_us":13344,"threads_started":1}
I20260806 22:38:34.832590 7405 tablet_service.cc:1467] Processing CreateTablet for tablet 64ed6a5ff98c4586bbee424444ef3d07 (DEFAULT_TABLE table=test-table [id=f07d00d19d594ae09e2de7a3dc98deef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:38:34.835592 7405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64ed6a5ff98c4586bbee424444ef3d07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:38:34.839975 7513 tablet_bootstrap.cc:492] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1: Bootstrap starting.
I20260806 22:38:34.842758 7515 tablet_bootstrap.cc:492] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2: Bootstrap starting.
I20260806 22:38:34.848384 7515 tablet_bootstrap.cc:654] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2: Neither blocks nor log segments found. Creating new log.
I20260806 22:38:34.850522 7515 log.cc:824] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:38:34.853938 7513 tablet_bootstrap.cc:654] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1: Neither blocks nor log segments found. Creating new log.
I20260806 22:38:34.856101 7515 tablet_bootstrap.cc:492] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2: No bootstrap required, opened a new log
I20260806 22:38:34.856031 7513 log.cc:824] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:38:34.856509 7515 ts_tablet_manager.cc:1397] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260806 22:38:34.859890 7518 tablet_bootstrap.cc:492] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0: Bootstrap starting.
I20260806 22:38:34.861006 7513 tablet_bootstrap.cc:492] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1: No bootstrap required, opened a new log
I20260806 22:38:34.861475 7513 ts_tablet_manager.cc:1397] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.003s
I20260806 22:38:34.864647 7518 tablet_bootstrap.cc:654] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0: Neither blocks nor log segments found. Creating new log.
I20260806 22:38:34.866132 7518 log.cc:824] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:38:34.868781 7515 raft_consensus.cc:348] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
I20260806 22:38:34.869395 7515 raft_consensus.cc:374] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:38:34.869529 7515 raft_consensus.cc:729] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a47243ada0194498925303cfa3d672a2, State: Initialized, Role: FOLLOWER
I20260806 22:38:34.870236 7518 tablet_bootstrap.cc:492] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0: No bootstrap required, opened a new log
I20260806 22:38:34.870362 7515 consensus_queue.cc:260] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
I20260806 22:38:34.870618 7518 ts_tablet_manager.cc:1397] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 22:38:34.871922 7316 heartbeater.cc:503] Master 127.15.198.190:33495 was elected leader, sending a full tablet report...
I20260806 22:38:34.873576 7513 raft_consensus.cc:348] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
I20260806 22:38:34.874164 7513 raft_consensus.cc:374] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:38:34.874374 7513 raft_consensus.cc:729] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9550c8e6ef8c4773bd0aa9b80355afc1, State: Initialized, Role: FOLLOWER
I20260806 22:38:34.875475 7515 ts_tablet_manager.cc:1428] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2: Time spent starting tablet: real 0.019s user 0.017s sys 0.000s
I20260806 22:38:34.875377 7513 consensus_queue.cc:260] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
I20260806 22:38:34.877832 7513 ts_tablet_manager.cc:1428] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260806 22:38:34.882874 7518 raft_consensus.cc:348] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
I20260806 22:38:34.883548 7518 raft_consensus.cc:374] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:38:34.883649 7518 raft_consensus.cc:729] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9be77cfb3ddc4803a453828afbf575e0, State: Initialized, Role: FOLLOWER
I20260806 22:38:34.884454 7518 consensus_queue.cc:260] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
I20260806 22:38:34.885802 7494 heartbeater.cc:503] Master 127.15.198.190:33495 was elected leader, sending a full tablet report...
I20260806 22:38:34.886744 7518 ts_tablet_manager.cc:1428] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
W20260806 22:38:34.951328 7495 tablet.cc:2369] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:38:34.970310 7522 raft_consensus.cc:482] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:38:34.970845 7522 raft_consensus.cc:504] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
W20260806 22:38:34.972492 7148 tablet.cc:2369] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:38:34.973845 7522 leader_election.cc:288] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a47243ada0194498925303cfa3d672a2 (127.15.198.130:45185), 9be77cfb3ddc4803a453828afbf575e0 (127.15.198.131:35351)
I20260806 22:38:34.982352 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:34.974471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:48667 (local address 127.15.198.130:45185)
0806 22:38:34.974797 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:34.974807 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:34.975253 (+ 446us) server_negotiation.cc:407] Connection header received
0806 22:38:34.975610 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:34.975617 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:34.975717 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:34.975801 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:38:34.976889 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.977923 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.980224 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.981123 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.981355 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:34.981479 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:34.981841 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 22:38:34.982026 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260806 22:38:34.982349 7512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:34.975448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:35983 (local address 127.15.198.131:35351)
0806 22:38:34.975720 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:34.975727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:34.975991 (+ 264us) server_negotiation.cc:407] Connection header received
0806 22:38:34.976184 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:34.976192 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:34.976291 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:34.976399 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:38:34.977221 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.978083 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.980207 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.981225 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.981496 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:34.981594 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:34.981882 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 22:38:34.982040 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260806 22:38:34.982528 7524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:34.974155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:45185 (local address 127.15.198.129:48667)
0806 22:38:34.974983 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:34.975109 (+ 126us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:34.975366 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:34.976094 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:34.976102 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:34.976132 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:38:34.976706 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:34.976721 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.978120 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.978129 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:34.980021 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:34.980040 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.981448 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.981463 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:34.981652 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:34.981682 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 22:38:34.981803 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 22:38:34.982004 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":99,"thread_start_us":445,"threads_started":1}
I20260806 22:38:34.983131 7525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:34.975111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:35351 (local address 127.15.198.129:35983)
0806 22:38:34.975883 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:34.975902 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:34.976040 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:34.976530 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:34.976537 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:34.976561 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:38:34.977037 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:34.977053 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.978236 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.978239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:34.980020 (+ 1781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:34.980035 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:34.981514 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:34.981520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:34.981715 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:34.981739 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 22:38:34.981849 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 22:38:34.982010 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":4,"thread_start_us":382,"threads_started":1}
I20260806 22:38:34.984122 7264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64ed6a5ff98c4586bbee424444ef3d07" candidate_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a47243ada0194498925303cfa3d672a2" is_pre_election: true
I20260806 22:38:34.984627 7433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64ed6a5ff98c4586bbee424444ef3d07" candidate_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9be77cfb3ddc4803a453828afbf575e0" is_pre_election: true
I20260806 22:38:34.984805 7264 raft_consensus.cc:2454] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9550c8e6ef8c4773bd0aa9b80355afc1 in term 0.
I20260806 22:38:34.985487 7433 raft_consensus.cc:2454] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9550c8e6ef8c4773bd0aa9b80355afc1 in term 0.
I20260806 22:38:34.985957 6990 leader_election.cc:302] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9550c8e6ef8c4773bd0aa9b80355afc1, a47243ada0194498925303cfa3d672a2; no voters:
I20260806 22:38:34.986584 7522 raft_consensus.cc:2781] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:38:34.986800 7522 raft_consensus.cc:482] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:38:34.986937 7522 raft_consensus.cc:3037] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:38:34.989791 7522 raft_consensus.cc:504] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
I20260806 22:38:34.990577 7522 leader_election.cc:288] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [CANDIDATE]: Term 1 election: Requested vote from peers a47243ada0194498925303cfa3d672a2 (127.15.198.130:45185), 9be77cfb3ddc4803a453828afbf575e0 (127.15.198.131:35351)
I20260806 22:38:34.991442 7264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64ed6a5ff98c4586bbee424444ef3d07" candidate_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a47243ada0194498925303cfa3d672a2"
I20260806 22:38:34.991608 7433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64ed6a5ff98c4586bbee424444ef3d07" candidate_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9be77cfb3ddc4803a453828afbf575e0"
I20260806 22:38:34.991734 7264 raft_consensus.cc:3037] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:38:34.991999 7433 raft_consensus.cc:3037] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:38:34.994720 7433 raft_consensus.cc:2454] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9550c8e6ef8c4773bd0aa9b80355afc1 in term 1.
I20260806 22:38:34.994716 7264 raft_consensus.cc:2454] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9550c8e6ef8c4773bd0aa9b80355afc1 in term 1.
I20260806 22:38:34.995373 6990 leader_election.cc:302] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9550c8e6ef8c4773bd0aa9b80355afc1, a47243ada0194498925303cfa3d672a2; no voters:
I20260806 22:38:34.995896 7522 raft_consensus.cc:2781] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:38:34.996476 7522 raft_consensus.cc:686] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 1 LEADER]: Becoming Leader. State: Replica: 9550c8e6ef8c4773bd0aa9b80355afc1, State: Running, Role: LEADER
I20260806 22:38:34.997095 7522 consensus_queue.cc:237] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } }
I20260806 22:38:35.003798 6895 catalog_manager.cc:5495] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9550c8e6ef8c4773bd0aa9b80355afc1 (127.15.198.129). New cstate: current_term: 1 leader_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9550c8e6ef8c4773bd0aa9b80355afc1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260806 22:38:35.027467 7509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:35.020633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41576 (local address 127.15.198.129:45045)
0806 22:38:35.020946 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:35.020953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:35.020981 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:38:35.021244 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:35.021249 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:35.021390 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:35.021616 (+ 226us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:38:35.022076 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:35.022959 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:35.024653 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:35.025141 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:35.025297 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:35.025996 (+ 699us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:38:35.026716 (+ 720us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:38:35.026913 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:35.026974 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 22:38:35.027125 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260806 22:38:35.039268 7433 raft_consensus.cc:1261] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0 [term 1 FOLLOWER]: Refusing update from remote peer 9550c8e6ef8c4773bd0aa9b80355afc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:38:35.039268 7264 raft_consensus.cc:1261] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2 [term 1 FOLLOWER]: Refusing update from remote peer 9550c8e6ef8c4773bd0aa9b80355afc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:38:35.040715 7526 consensus_queue.cc:1035] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a47243ada0194498925303cfa3d672a2" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:38:35.041273 7522 consensus_queue.cc:1035] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9be77cfb3ddc4803a453828afbf575e0" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:38:35.061672 7533 mvcc.cc:205] Tried to move back new op lower bound from 7315685027986006016 to 7315685027833008128. Current Snapshot: MvccSnapshot[applied={T|T < 7315685027986006016}]
I20260806 22:38:35.080909 6896 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:38:35.081460 6896 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:38:35.086704 6896 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Sending DeleteTablet for 3 replicas of tablet 64ed6a5ff98c4586bbee424444ef3d07
I20260806 22:38:35.088505 7237 tablet_service.cc:1514] Processing DeleteTablet for tablet 64ed6a5ff98c4586bbee424444ef3d07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:38:35 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:58430
I20260806 22:38:35.088505 7060 tablet_service.cc:1514] Processing DeleteTablet for tablet 64ed6a5ff98c4586bbee424444ef3d07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:38:35 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:41566
I20260806 22:38:35.091722 7405 tablet_service.cc:1514] Processing DeleteTablet for tablet 64ed6a5ff98c4586bbee424444ef3d07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:38:35 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:46404
I20260806 22:38:35.100651 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 6943
I20260806 22:38:35.105156 7546 tablet_replica.cc:330] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1: stopping tablet replica
I20260806 22:38:35.106040 7549 tablet_replica.cc:330] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0: stopping tablet replica
I20260806 22:38:35.109236 7549 raft_consensus.cc:2229] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:38:35.110234 7546 raft_consensus.cc:2229] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:38:35.112538 7547 tablet_replica.cc:330] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2: stopping tablet replica
I20260806 22:38:35.113152 7549 raft_consensus.cc:2258] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:38:35.113234 7547 raft_consensus.cc:2229] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:38:35.113958 7546 raft_consensus.cc:2258] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:38:35.114049 7547 raft_consensus.cc:2258] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:38:35.117300 7549 ts_tablet_manager.cc:1905] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:38:35.119215 7547 ts_tablet_manager.cc:1905] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:38:35.124178 7546 ts_tablet_manager.cc:1905] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:38:35.126698 7549 ts_tablet_manager.cc:1918] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:38:35.126937 7549 log.cc:1196] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/64ed6a5ff98c4586bbee424444ef3d07
I20260806 22:38:35.127280 7509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:35.102409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41586 (local address 127.15.198.129:45045)
0806 22:38:35.102718 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:35.102727 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:35.102756 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:38:35.103201 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:35.103222 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:35.103468 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:35.103857 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:35.106643 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:35.108472 (+ 1829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:35.110812 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:35.111706 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:35.111876 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:35.117144 (+ 5268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:35.117167 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:35.117189 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:35.117256 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:35.122498 (+ 5242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:35.123937 (+ 1439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:35.123953 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:35.123959 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:35.124191 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:35.124716 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:35.124722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:35.124725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:35.125355 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:35.125749 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:35.126117 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 22:38:35.126501 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260806 22:38:35.127691 7549 ts_tablet_manager.cc:1939] T 64ed6a5ff98c4586bbee424444ef3d07 P 9be77cfb3ddc4803a453828afbf575e0: Deleting consensus metadata
I20260806 22:38:35.128646 7141 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:38:35.130386 6876 catalog_manager.cc:4841] TS 9be77cfb3ddc4803a453828afbf575e0 (127.15.198.131:35351): tablet 64ed6a5ff98c4586bbee424444ef3d07 (table test-table [id=f07d00d19d594ae09e2de7a3dc98deef]) successfully deleted
I20260806 22:38:35.135262 7547 ts_tablet_manager.cc:1918] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:38:35.135568 7547 log.cc:1196] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/64ed6a5ff98c4586bbee424444ef3d07
I20260806 22:38:35.136204 7547 ts_tablet_manager.cc:1939] T 64ed6a5ff98c4586bbee424444ef3d07 P a47243ada0194498925303cfa3d672a2: Deleting consensus metadata
I20260806 22:38:35.138392 6876 catalog_manager.cc:4841] TS a47243ada0194498925303cfa3d672a2 (127.15.198.130:45185): tablet 64ed6a5ff98c4586bbee424444ef3d07 (table test-table [id=f07d00d19d594ae09e2de7a3dc98deef]) successfully deleted
I20260806 22:38:35.246439 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 6943
W20260806 22:38:35.285362 6878 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.198.129:45045 (error 108)
I20260806 22:38:35.285818 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 7153
I20260806 22:38:35.288096 7507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:35.287396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:45045 (local address 127.0.0.1:41594)
0806 22:38:35.287648 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:35.287778 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260806 22:38:35.288445 6878 catalog_manager.cc:4584] TS 9550c8e6ef8c4773bd0aa9b80355afc1 (127.15.198.129:45045): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 64ed6a5ff98c4586bbee424444ef3d07: Network error: Client connection negotiation failed: client connection to 127.15.198.129:45045: connect: Connection refused (error 111)
I20260806 22:38:35.304636 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:35.286775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58440 (local address 127.15.198.130:45185)
0806 22:38:35.287033 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:35.287040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:35.287067 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:38:35.287433 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:35.287442 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:35.287735 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:35.287900 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:35.290377 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:35.291393 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:35.293114 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:35.293584 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:35.293697 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:35.297037 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:35.297059 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:35.297081 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:35.297131 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:35.300948 (+ 3817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:35.301989 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:35.302000 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:35.302003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:35.302166 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:35.302547 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:35.302550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:35.302552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:35.303117 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:35.303407 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:35.303858 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 22:38:35.304125 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:38:35.305486 7311 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:38:35.342941 7507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:35.342218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:45045 (local address 127.0.0.1:41604)
0806 22:38:35.342513 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:35.342653 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260806 22:38:35.344444 7507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:35.343907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:45045 (local address 127.0.0.1:41620)
0806 22:38:35.344111 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:35.344220 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 22:38:35.405601 7507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:35.404907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:45045 (local address 127.0.0.1:41636)
0806 22:38:35.405193 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:35.405336 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260806 22:38:35.407058 7507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:35.406500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:45045 (local address 127.0.0.1:41650)
0806 22:38:35.406690 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:35.406795 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 22:38:35.411852 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 7153
I20260806 22:38:35.443606 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 7321
I20260806 22:38:35.461193 7512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:35.444663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46412 (local address 127.15.198.131:35351)
0806 22:38:35.444871 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:35.444876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:35.444900 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:38:35.445113 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:35.445118 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:35.445202 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:35.445337 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:35.447122 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:35.448074 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:35.449587 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:35.450121 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:35.450242 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:35.453754 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:35.453783 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:35.453810 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:35.453859 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:35.457631 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:35.458697 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:35.458708 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:35.458710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:35.458909 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:35.459421 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:35.459425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:35.459427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:35.459907 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:35.460248 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:35.460587 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 22:38:35.460819 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 22:38:35.462059 7486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:38:35.488497 7507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:35.487762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:45045 (local address 127.0.0.1:41652)
0806 22:38:35.488072 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:35.488218 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260806 22:38:35.490170 7507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:35.489503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:45045 (local address 127.0.0.1:41658)
0806 22:38:35.489788 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:35.489931 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260806 22:38:35.570149 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 7321
I20260806 22:38:35.607854 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 6857
I20260806 22:38:35.609894 6933 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:38:35.750054 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 6857
I20260806 22:38:55.790458 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:33495
--webserver_interface=127.15.198.190
--webserver_port=38655
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:42223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:33495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:38:56.257903 7557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:56.258424 7557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:56.258620 7557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:56.268100 7557 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:38:56.268307 7557 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:38:56.268417 7557 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:38:56.268529 7557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:38:56.268639 7557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:56.268786 7557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:56.268872 7557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:38:56.268968 7557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:38:56.280810 7557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:42223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:33495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:33495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=38655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:38:56.282819 7557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:56.284739 7557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:56.300679 7568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:56.300643 7564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:56.301213 7557 server_base.cc:1034] running on GCE node
W20260806 22:38:56.300722 7563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:56.302863 7557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:56.304935 7557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:56.306249 7557 hybrid_clock.cc:648] HybridClock initialized: now 1786055936306165 us; error 64 us; skew 500 ppm
I20260806 22:38:56.306713 7557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:38:56 dist-test-slave-h3ft krb5kdc[6115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055936, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:38:56.316627 7557 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:38:56.318944 7557 webserver.cc:460] Webserver started at http://127.15.198.190:38655/ using document root <none> and password file <none>
I20260806 22:38:56.320008 7557 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:56.320140 7557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:56.327615 7557 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 22:38:56.331642 7579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:56.333640 7557 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:38:56.333957 7557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "c042426a636e498b9996a841635e7dff"
format_stamp: "Formatted at 2026-08-06 22:38:08 on dist-test-slave-h3ft"
server_key: "a548424a82b091aa7dba34c3524f8931"
server_key_iv: "1a1eb6d0b5b3265a545e1c8daa1b7d68"
server_key_version: "encryptionkey@0"
I20260806 22:38:56.335332 7557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:56.362131 7557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:56.368232 7557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:56.368696 7557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:56.395588 7557 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:33495
I20260806 22:38:56.395613 7648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:33495 every 8 connection(s)
I20260806 22:38:56.397817 7557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:38:56.400521 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 7557
I20260806 22:38:56.401573 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:45045
--local_ip_for_outbound_sockets=127.15.198.129
--tserver_master_addrs=127.15.198.190:33495
--webserver_port=33005
--webserver_interface=127.15.198.129
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:42223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:38:56.407475 7649 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:38:56.411697 7649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Bootstrap starting.
I20260806 22:38:56.437776 7649 log.cc:824] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Log is configured to *not* fsync() on all Append() calls
I20260806 22:38:56.458266 7649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:38:56.459118 7649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Bootstrap complete.
I20260806 22:38:56.470219 7649 raft_consensus.cc:348] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } }
I20260806 22:38:56.471482 7649 raft_consensus.cc:729] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c042426a636e498b9996a841635e7dff, State: Initialized, Role: FOLLOWER
I20260806 22:38:56.472308 7649 consensus_queue.cc:260] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } }
I20260806 22:38:56.472666 7649 raft_consensus.cc:388] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:38:56.472848 7649 raft_consensus.cc:482] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:38:56.473078 7649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:38:56.478189 7649 raft_consensus.cc:504] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } }
I20260806 22:38:56.478932 7649 leader_election.cc:302] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c042426a636e498b9996a841635e7dff; no voters:
I20260806 22:38:56.479897 7649 leader_election.cc:288] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 22:38:56.480065 7653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:38:56.481472 7653 raft_consensus.cc:686] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [term 3 LEADER]: Becoming Leader. State: Replica: c042426a636e498b9996a841635e7dff, State: Running, Role: LEADER
I20260806 22:38:56.482300 7653 consensus_queue.cc:237] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } }
I20260806 22:38:56.482803 7649 sys_catalog.cc:564] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:38:56.485584 7659 sys_catalog.cc:455] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [sys.catalog]: SysCatalogTable state changed. Reason: New leader c042426a636e498b9996a841635e7dff. Latest consensus state: current_term: 3 leader_uuid: "c042426a636e498b9996a841635e7dff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } } }
I20260806 22:38:56.486117 7659 sys_catalog.cc:458] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [sys.catalog]: This master's current role is: LEADER
I20260806 22:38:56.486023 7655 sys_catalog.cc:455] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c042426a636e498b9996a841635e7dff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c042426a636e498b9996a841635e7dff" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33495 } } }
I20260806 22:38:56.486508 7655 sys_catalog.cc:458] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff [sys.catalog]: This master's current role is: LEADER
I20260806 22:38:56.497359 7666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:38:56.510737 7666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:38:56.513769 7666 catalog_manager.cc:1259] Loaded cluster ID: 335287833ba049098ee01aa86dbdfa52
I20260806 22:38:56.514073 7666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:38:56.520539 7666 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:38:56.524152 7666 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Loaded TSK: 0
I20260806 22:38:56.524503 7666 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c042426a636e498b9996a841635e7dff: Loaded TSK: 1
F20260806 22:38:56.526024 7666 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe52e9e59ad google::LogMessage::Fail() at ??:0
@ 0x7fe52e9e888a google::LogMessage::SendToLog() at ??:0
@ 0x7fe52e9e54de google::LogMessage::Flush() at ??:0
@ 0x7fe52e9e6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe5327497d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fe53273873e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fe5444141c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fe544413783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fe54449f2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fe54449f0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fe5444e95e9 std::function<>::operator()() at ??:0
@ 0x7fe544417cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fe544415c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe54449d610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fe54449d439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe531631345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe53163491c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fe531634799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe53160feaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe5348e96db start_thread at ??:0
@ 0x7fe52d87f61f clone at ??:0
W20260806 22:38:56.916042 7651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:56.916538 7651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:56.916734 7651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:56.925865 7651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:56.926122 7651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:56.926369 7651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:38:56.938913 7651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:42223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:45045
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=33005
--tserver_master_addrs=127.15.198.190:33495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:38:56.941455 7651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:56.943485 7651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:56.960927 7717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:56.960880 7713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:56.960955 7712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:56.963351 7651 server_base.cc:1034] running on GCE node
I20260806 22:38:56.964349 7651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:56.965778 7651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:56.967163 7651 hybrid_clock.cc:648] HybridClock initialized: now 1786055936967103 us; error 49 us; skew 500 ppm
I20260806 22:38:56.967636 7651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:38:56 dist-test-slave-h3ft krb5kdc[6115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055936, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:38:56.972018 7651 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:38:56.974440 7651 webserver.cc:460] Webserver started at http://127.15.198.129:33005/ using document root <none> and password file <none>
I20260806 22:38:56.975473 7651 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:56.975618 7651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:56.982718 7651 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 22:38:56.986654 7728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:56.988531 7651 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:38:56.988729 7651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "9550c8e6ef8c4773bd0aa9b80355afc1"
format_stamp: "Formatted at 2026-08-06 22:38:09 on dist-test-slave-h3ft"
server_key: "1e8598c15a40939ad23b1f20a17be967"
server_key_iv: "e4a690c3ddaec41eeef9100e018668a7"
server_key_version: "encryptionkey@0"
I20260806 22:38:56.989879 7651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:57.017678 7651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:57.023386 7651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:57.023699 7651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:57.024931 7651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:57.027490 7739 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260806 22:38:57.034569 7739 ts_tablet_manager.cc:1852] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260806 22:38:57.035809 7739 ts_tablet_manager.cc:1905] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:38:57.045281 7739 ts_tablet_manager.cc:1918] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:38:57.045584 7739 log.cc:1196] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/64ed6a5ff98c4586bbee424444ef3d07
I20260806 22:38:57.046226 7739 ts_tablet_manager.cc:1939] T 64ed6a5ff98c4586bbee424444ef3d07 P 9550c8e6ef8c4773bd0aa9b80355afc1: Deleting consensus metadata
I20260806 22:38:57.046733 7651 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260806 22:38:57.046901 7651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.020s user 0.001s sys 0.000s
I20260806 22:38:57.047128 7651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:57.047250 7651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:57.093058 7651 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:45045
I20260806 22:38:57.093107 7880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:45045 every 8 connection(s)
I20260806 22:38:57.094939 7651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:38:57.098138 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 7651
I20260806 22:38:57.098964 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:45185
--local_ip_for_outbound_sockets=127.15.198.130
--tserver_master_addrs=127.15.198.190:33495
--webserver_port=45409
--webserver_interface=127.15.198.130
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:42223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:38:57.102344 7883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:57.099229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:33585)
0806 22:38:57.100377 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:57.100762 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":383,"threads_started":1}
W20260806 22:38:57.103399 7881 heartbeater.cc:644] Failed to heartbeat to 127.15.198.190:33495 (0 consecutive failures): Network error: Failed to ping master at 127.15.198.190:33495: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
I20260806 22:38:57.104818 7883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:57.104270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:40431)
0806 22:38:57.104510 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:57.104636 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 22:38:57.106056 7883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:57.105573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:58877)
0806 22:38:57.105761 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:57.105874 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 22:38:57.106412 7881 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:38:57.543495 7884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:57.544148 7884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:57.544350 7884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:57.553232 7884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:57.553406 7884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:57.553625 7884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:38:57.565478 7884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:42223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:45185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=45409
--tserver_master_addrs=127.15.198.190:33495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:38:57.567515 7884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:57.569346 7884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:57.586645 7892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:57.586592 7893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:57.586616 7895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:57.587702 7884 server_base.cc:1034] running on GCE node
I20260806 22:38:57.588539 7884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:57.589843 7884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:57.591173 7884 hybrid_clock.cc:648] HybridClock initialized: now 1786055937591108 us; error 52 us; skew 500 ppm
I20260806 22:38:57.591583 7884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:38:57 dist-test-slave-h3ft krb5kdc[6115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055937, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:38:57.596022 7884 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:38:57.598116 7884 webserver.cc:460] Webserver started at http://127.15.198.130:45409/ using document root <none> and password file <none>
I20260806 22:38:57.599148 7884 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:57.599323 7884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:57.607802 7884 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 22:38:57.612030 7905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:57.614071 7884 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:38:57.614312 7884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "a47243ada0194498925303cfa3d672a2"
format_stamp: "Formatted at 2026-08-06 22:38:10 on dist-test-slave-h3ft"
server_key: "a4ef785b9c7ae7dc05becfa2a84e69df"
server_key_iv: "d76f8f362b9a62cc146c20f2c10e0ed7"
server_key_version: "encryptionkey@0"
I20260806 22:38:57.615849 7884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:57.648470 7884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:57.654277 7884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:57.654693 7884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:57.656265 7884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:57.658226 7884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:38:57.658339 7884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:57.658572 7884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:57.658687 7884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:57.702382 7884 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:45185
I20260806 22:38:57.702454 8063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:45185 every 8 connection(s)
I20260806 22:38:57.704526 7884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:38:57.705446 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 7884
I20260806 22:38:57.706470 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:35351
--local_ip_for_outbound_sockets=127.15.198.131
--tserver_master_addrs=127.15.198.190:33495
--webserver_port=34829
--webserver_interface=127.15.198.131
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:42223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:38:57.712366 8067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:57.709139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:45461)
0806 22:38:57.710229 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:57.710614 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":409,"threads_started":1}
W20260806 22:38:57.713258 8065 heartbeater.cc:644] Failed to heartbeat to 127.15.198.190:33495 (0 consecutive failures): Network error: Failed to ping master at 127.15.198.190:33495: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
I20260806 22:38:57.714574 8067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:57.714034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:35015)
0806 22:38:57.714265 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:57.714378 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 22:38:57.715775 8067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:57.715293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:53009)
0806 22:38:57.715479 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:57.715595 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 22:38:57.716130 8065 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:38:58.108567 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:58.107365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:50823)
0806 22:38:58.108101 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:58.108238 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":381,"threads_started":1}
W20260806 22:38:58.159518 8070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:58.159879 8070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:58.160090 8070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:58.168895 8070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:58.169061 8070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:58.169273 8070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:38:58.180823 8070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:42223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:35351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=34829
--tserver_master_addrs=127.15.198.190:33495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:38:58.182739 8070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:58.184577 8070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:58.201084 8081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:58.201138 8079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:58.201020 8077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:58.202790 8070 server_base.cc:1034] running on GCE node
I20260806 22:38:58.203723 8070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:58.204999 8070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:58.206288 8070 hybrid_clock.cc:648] HybridClock initialized: now 1786055938206201 us; error 73 us; skew 500 ppm
I20260806 22:38:58.206745 8070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:38:58 dist-test-slave-h3ft krb5kdc[6115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055938, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:38:58.210675 8070 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:38:58.212698 8070 webserver.cc:460] Webserver started at http://127.15.198.131:34829/ using document root <none> and password file <none>
I20260806 22:38:58.213608 8070 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:58.213778 8070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:58.220680 8070 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:38:58.224617 8090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:58.226632 8070 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 22:38:58.226910 8070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "9be77cfb3ddc4803a453828afbf575e0"
format_stamp: "Formatted at 2026-08-06 22:38:10 on dist-test-slave-h3ft"
server_key: "0a11f693e298c103cc8c88ebe9517c0b"
server_key_iv: "696e496a060592b31cab323880fca91a"
server_key_version: "encryptionkey@0"
I20260806 22:38:58.228416 8070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:58.246727 8070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:58.252522 8070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:58.252910 8070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:58.254240 8070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:58.256117 8070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:38:58.256256 8070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:58.256414 8070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:58.256515 8070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:58.298429 8070 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:35351
I20260806 22:38:58.298476 8250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:35351 every 8 connection(s)
I20260806 22:38:58.300468 8070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:38:58.304185 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 8070
I20260806 22:38:58.307293 8253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:58.304411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:53127)
0806 22:38:58.305630 (+ 1219us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:58.306000 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":770,"thread_start_us":446,"threads_started":1}
W20260806 22:38:58.308121 8251 heartbeater.cc:644] Failed to heartbeat to 127.15.198.190:33495 (0 consecutive failures): Network error: Failed to ping master at 127.15.198.190:33495: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
I20260806 22:38:58.309055 8253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:58.308701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:48715)
0806 22:38:58.308859 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:58.308939 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 22:38:58.310088 8253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:58.309652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:58659)
0806 22:38:58.309828 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:58.309934 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 22:38:58.310443 8251 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:38:58.718079 8255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:58.716973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:40341)
0806 22:38:58.717619 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:58.717740 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":325,"threads_started":1}
I20260806 22:38:59.111294 8256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:59.110171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:49045)
0806 22:38:59.110920 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:59.111082 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":431,"threads_started":1}
I20260806 22:38:59.312395 8257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:59.311343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:42859)
0806 22:38:59.311956 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:59.312096 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":283,"threads_started":1}
I20260806 22:38:59.720463 8258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:59.719481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:45629)
0806 22:38:59.720104 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:59.720222 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":298,"threads_started":1}
I20260806 22:39:00.113555 8260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:00.112528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:43635)
0806 22:39:00.113176 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:00.113297 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":310,"threads_started":1}
I20260806 22:39:00.314744 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:00.313766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:43517)
0806 22:39:00.314399 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:00.314522 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":334,"threads_started":1}
I20260806 22:39:00.722966 8263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:00.721695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:40881)
0806 22:39:00.722523 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:00.722662 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":506,"threads_started":1}
I20260806 22:39:01.116431 8264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:01.115186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:34327)
0806 22:39:01.115912 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:01.116075 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":399,"threads_started":1}
I20260806 22:39:01.316834 8265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:01.315931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:36893)
0806 22:39:01.316494 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:01.316612 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":317,"threads_started":1}
I20260806 22:39:01.725468 8266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:01.724349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:34821)
0806 22:39:01.725068 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:01.725212 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":394,"threads_started":1}
I20260806 22:39:02.119107 8267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:02.117974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:39433)
0806 22:39:02.118614 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:02.118726 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":327,"threads_started":1}
I20260806 22:39:02.319506 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:02.318324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:40329)
0806 22:39:02.319157 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:02.319269 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":407,"threads_started":1}
I20260806 22:39:02.728003 8269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:02.726925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:47981)
0806 22:39:02.727633 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:02.727758 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":387,"threads_started":1}
I20260806 22:39:03.121938 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:03.120735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:47965)
0806 22:39:03.121455 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:03.121594 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":376,"threads_started":1}
I20260806 22:39:03.322021 8271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:03.320888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:40943)
0806 22:39:03.321648 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:03.321770 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":317,"threads_started":1}
I20260806 22:39:03.730830 8272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:03.729550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:35155)
0806 22:39:03.730352 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:03.730556 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":444,"threads_started":1}
I20260806 22:39:04.124472 8273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.123248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:36151)
0806 22:39:04.124082 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:04.124191 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":431,"threads_started":1}
I20260806 22:39:04.324322 8274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.323345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:39787)
0806 22:39:04.323963 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:04.324095 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":288,"threads_started":1}
I20260806 22:39:04.734156 8275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.732528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:37465)
0806 22:39:04.733561 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:04.733712 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":745,"thread_start_us":404,"threads_started":1}
I20260806 22:39:05.127235 8276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:05.125922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:56299)
0806 22:39:05.126726 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:05.126930 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":418,"threads_started":1}
I20260806 22:39:05.327324 8278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:05.325768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:56607)
0806 22:39:05.326763 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:05.326935 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":827,"thread_start_us":369,"threads_started":1}
I20260806 22:39:05.736949 8280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:05.735780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:33119)
0806 22:39:05.736582 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:05.736702 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":465,"threads_started":1}
I20260806 22:39:06.129705 8282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:06.128624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:38323)
0806 22:39:06.129288 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:06.129452 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":339,"threads_started":1}
I20260806 22:39:06.329947 8283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:06.328815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:41297)
0806 22:39:06.329565 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:06.329702 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":376,"threads_started":1}
I20260806 22:39:06.739886 8285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:06.738527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:35659)
0806 22:39:06.739387 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:06.739522 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":534,"threads_started":1}
I20260806 22:39:07.132458 8286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:07.131333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:44407)
0806 22:39:07.132078 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:07.132215 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":341,"threads_started":1}
I20260806 22:39:07.332392 8287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:07.331334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:49267)
0806 22:39:07.332039 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:07.332156 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":364,"threads_started":1}
I20260806 22:39:07.742595 8288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:07.741423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:57215)
0806 22:39:07.742149 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:07.742314 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":401,"threads_started":1}
I20260806 22:39:08.134990 8289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:08.133931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:60967)
0806 22:39:08.134589 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:08.134703 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":378,"threads_started":1}
I20260806 22:39:08.334796 8290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:08.333794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:56323)
0806 22:39:08.334470 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:08.334579 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":341,"threads_started":1}
I20260806 22:39:08.745085 8291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:08.743989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:50439)
0806 22:39:08.744703 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:08.744830 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":377,"threads_started":1}
I20260806 22:39:09.137358 8292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:09.136342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:46709)
0806 22:39:09.137003 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:09.137123 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":355,"threads_started":1}
I20260806 22:39:09.337138 8293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:09.336024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:44171)
0806 22:39:09.336732 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:09.336855 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":371,"threads_started":1}
I20260806 22:39:09.747449 8294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:09.746410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:50113)
0806 22:39:09.747079 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:09.747206 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":313,"threads_started":1}
I20260806 22:39:10.140003 8295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:10.138735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:58459)
0806 22:39:10.139529 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:10.139730 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":471,"threads_started":1}
I20260806 22:39:10.339782 8296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:10.338649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:34059)
0806 22:39:10.339349 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:10.339492 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":388,"threads_started":1}
I20260806 22:39:10.750286 8297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:10.749048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:39411)
0806 22:39:10.749772 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:10.749986 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":395,"threads_started":1}
I20260806 22:39:11.142247 8298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:11.141268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:54353)
0806 22:39:11.141868 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:11.141983 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":308,"threads_started":1}
I20260806 22:39:11.341969 8299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:11.340958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:33407)
0806 22:39:11.341606 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:11.341726 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":347,"threads_started":1}
I20260806 22:39:11.752940 8300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:11.751737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:44849)
0806 22:39:11.752481 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:11.752643 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":431,"threads_started":1}
I20260806 22:39:12.145246 8302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:12.144146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:60413)
0806 22:39:12.144786 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:12.144972 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":292,"threads_started":1}
I20260806 22:39:12.344480 8303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:12.343439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:49405)
0806 22:39:12.344122 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:12.344246 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":320,"threads_started":1}
I20260806 22:39:12.755611 8305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:12.754468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.130:47013)
0806 22:39:12.755202 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:12.755356 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":381,"threads_started":1}
I20260806 22:39:13.147696 8307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:13.146613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.129:53653)
0806 22:39:13.147358 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:13.147479 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":430,"threads_started":1}
I20260806 22:39:13.315781 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 7651
I20260806 22:39:13.357173 8309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:13.346101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33495 (local address 127.15.198.131:41325)
0806 22:39:13.356595 (+ 10494us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:13.356785 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.190:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10314,"thread_start_us":10149,"threads_started":1}
I20260806 22:39:13.359175 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:13.317485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59844 (local address 127.15.198.129:45045)
0806 22:39:13.318470 (+ 985us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:13.318496 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:13.318713 (+ 217us) server_negotiation.cc:407] Connection header received
0806 22:39:13.321190 (+ 2477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:13.321202 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:13.321796 (+ 594us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:13.322308 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:13.324269 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:13.325736 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:13.327288 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:13.328083 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:13.328321 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:13.343241 (+ 14920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:13.343266 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:13.343306 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:13.343401 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:13.351632 (+ 8231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:13.353218 (+ 1586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:13.353236 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:13.353254 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:13.353616 (+ 362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:13.354312 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:13.354321 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:13.354328 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:13.355612 (+ 1284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:13.357654 (+ 2042us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:13.358134 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 22:39:13.358683 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":478,"threads_started":1}
I20260806 22:39:13.362272 7874 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:13.475455 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 7651
I20260806 22:39:13.508468 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 7884
I20260806 22:39:13.532281 8311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:13.509843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47714 (local address 127.15.198.130:45185)
0806 22:39:13.510582 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:13.510602 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:13.510652 (+ 50us) server_negotiation.cc:407] Connection header received
0806 22:39:13.512904 (+ 2252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:13.512919 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:13.513558 (+ 639us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:13.513961 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:13.515906 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:13.517430 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:13.518844 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:13.519735 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:13.519902 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:13.523397 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:13.523409 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:13.523433 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:13.523491 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:13.527761 (+ 4270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:13.528725 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:13.528736 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:13.528745 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:13.529022 (+ 277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:13.529612 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:13.529617 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:13.529620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:13.530455 (+ 835us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:13.530763 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:13.531384 (+ 621us) server_negotiation.cc:299] Negotiation successful
0806 22:39:13.531869 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":382,"threads_started":1}
I20260806 22:39:13.534360 8055 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:13.629376 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 7884
I20260806 22:39:13.662063 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 8070
I20260806 22:39:13.686726 8313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:13.663679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36926 (local address 127.15.198.131:35351)
0806 22:39:13.664483 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:13.664502 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:13.664558 (+ 56us) server_negotiation.cc:407] Connection header received
0806 22:39:13.666560 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:13.666573 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:13.667292 (+ 719us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:13.667709 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:13.669558 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:13.671348 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:13.672792 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:13.673613 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:13.673788 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:13.677205 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:13.677222 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:13.677254 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:13.677327 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:13.682036 (+ 4709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:13.682874 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:13.682883 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:13.682892 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:13.683227 (+ 335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:13.683659 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:13.683663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:13.683666 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:13.684622 (+ 956us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:13.685029 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:13.685651 (+ 622us) server_negotiation.cc:299] Negotiation successful
0806 22:39:13.686305 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":388,"threads_started":1}
I20260806 22:39:13.688805 8245 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:13.784623 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 8070
2026-08-06T22:39:13Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (67454 ms)
[----------] 30 tests from SecurityITest (357800 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:39:13 dist-test-slave-h3ft krb5kdc[8318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:39:13 dist-test-slave-h3ft krb5kdc[8318](info): set up 2 sockets
Aug 06 22:39:13 dist-test-slave-h3ft krb5kdc[8318](info): commencing operation
krb5kdc: starting...
W20260806 22:39:15.220590 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.378s user 0.000s sys 0.065s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:39:15 dist-test-slave-h3ft krb5kdc[8318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:39:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:39:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:39:15.655661 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:36841
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:37945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:36841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:16.122923 8335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:16.123498 8335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:16.123612 8335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:16.132000 8335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:16.132154 8335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:16.132227 8335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:16.132285 8335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:16.132334 8335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:16.143846 8335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:37945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:36841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:36841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:16.145754 8335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:16.147526 8335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:16.163040 8340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:16.162943 8341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:16.163115 8344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:16.164232 8335 server_base.cc:1034] running on GCE node
I20260806 22:39:16.165359 8335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:16.167421 8335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:16.168720 8335 hybrid_clock.cc:648] HybridClock initialized: now 1786055956168641 us; error 65 us; skew 500 ppm
I20260806 22:39:16.169217 8335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:16 dist-test-slave-h3ft krb5kdc[8318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055956, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:16.173696 8335 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:39:16.176083 8335 webserver.cc:460] Webserver started at http://127.15.198.190:44013/ using document root <none> and password file <none>
I20260806 22:39:16.177027 8335 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:16.177156 8335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:16.177496 8335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:16.181589 8335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "76ccf26159834bb6b4d6c6dca2ae5a56"
format_stamp: "Formatted at 2026-08-06 22:39:16 on dist-test-slave-h3ft"
server_key: "57f62ed4fa1a23ddc28105d7c507148c"
server_key_iv: "8af8564312bd5592c12d50a10ceb660b"
server_key_version: "encryptionkey@0"
I20260806 22:39:16.182515 8335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "76ccf26159834bb6b4d6c6dca2ae5a56"
format_stamp: "Formatted at 2026-08-06 22:39:16 on dist-test-slave-h3ft"
server_key: "57f62ed4fa1a23ddc28105d7c507148c"
server_key_iv: "8af8564312bd5592c12d50a10ceb660b"
server_key_version: "encryptionkey@0"
I20260806 22:39:16.189584 8335 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 22:39:16.194244 8351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:16.196168 8335 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 22:39:16.196373 8335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "76ccf26159834bb6b4d6c6dca2ae5a56"
format_stamp: "Formatted at 2026-08-06 22:39:16 on dist-test-slave-h3ft"
server_key: "57f62ed4fa1a23ddc28105d7c507148c"
server_key_iv: "8af8564312bd5592c12d50a10ceb660b"
server_key_version: "encryptionkey@0"
I20260806 22:39:16.196599 8335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:16.226763 8335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:16.232300 8335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:16.232628 8335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:16.255838 8335 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:36841
I20260806 22:39:16.255873 8424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:36841 every 8 connection(s)
I20260806 22:39:16.257717 8335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:39:16.263145 8425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:16.263414 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 8335
I20260806 22:39:16.263690 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:39:16.264672 16154 external_mini_cluster.cc:1428] Setting key 7ddc04fed03009f7e8ab2ffdef2d3ea6
I20260806 22:39:16.281013 8425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56: Bootstrap starting.
Aug 06 22:39:16 dist-test-slave-h3ft krb5kdc[8318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:39:16.285889 8425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:16.287384 8425 log.cc:824] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:16.291587 8425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56: No bootstrap required, opened a new log
I20260806 22:39:16.296103 8430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:16.267480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36802 (local address 127.15.198.190:36841)
0806 22:39:16.268478 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:16.268494 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:16.268551 (+ 57us) server_negotiation.cc:407] Connection header received
0806 22:39:16.271128 (+ 2577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:16.271161 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:16.271745 (+ 584us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:16.272183 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:16.274316 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:16.276133 (+ 1817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:16.278406 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:16.279438 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:16.279603 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:16.285689 (+ 6086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:16.285701 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:16.285721 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:16.285781 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:16.290128 (+ 4347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:16.291104 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:16.291115 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:16.291124 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:16.291396 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:16.291965 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:16.291971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:16.291974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:16.292860 (+ 886us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:16.293218 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:16.293790 (+ 572us) server_negotiation.cc:299] Negotiation successful
0806 22:39:16.294398 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":479,"threads_started":1}
I20260806 22:39:16.301749 8425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ccf26159834bb6b4d6c6dca2ae5a56" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36841 } }
I20260806 22:39:16.302243 8425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:16.302371 8425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76ccf26159834bb6b4d6c6dca2ae5a56, State: Initialized, Role: FOLLOWER
I20260806 22:39:16.303144 8425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ccf26159834bb6b4d6c6dca2ae5a56" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36841 } }
I20260806 22:39:16.303494 8425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:39:16.303650 8425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:39:16.303864 8425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:16.307332 8425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ccf26159834bb6b4d6c6dca2ae5a56" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36841 } }
I20260806 22:39:16.308132 8425 leader_election.cc:302] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76ccf26159834bb6b4d6c6dca2ae5a56; no voters:
I20260806 22:39:16.308785 8425 leader_election.cc:288] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:39:16.308980 8432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:16.309850 8432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [term 1 LEADER]: Becoming Leader. State: Replica: 76ccf26159834bb6b4d6c6dca2ae5a56, State: Running, Role: LEADER
I20260806 22:39:16.310477 8432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ccf26159834bb6b4d6c6dca2ae5a56" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36841 } }
I20260806 22:39:16.311631 8425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:16.314433 8435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76ccf26159834bb6b4d6c6dca2ae5a56. Latest consensus state: current_term: 1 leader_uuid: "76ccf26159834bb6b4d6c6dca2ae5a56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ccf26159834bb6b4d6c6dca2ae5a56" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36841 } } }
I20260806 22:39:16.314990 8435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:16.317360 8433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76ccf26159834bb6b4d6c6dca2ae5a56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ccf26159834bb6b4d6c6dca2ae5a56" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36841 } } }
I20260806 22:39:16.317785 8433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56 [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:16.320138 8438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:39:16.326490 8438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:39:16.340484 8438 catalog_manager.cc:1347] Generated new cluster ID: 0d58c2de90664a14bbd41bd3c26129d7
I20260806 22:39:16.340658 8438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:39:16.361340 8438 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:39:16.362975 8438 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:39:16.372893 8438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56: Generated new TSK 0
I20260806 22:39:16.374017 8438 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
W20260806 22:39:16.487039 16154 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.198.129: real 0.102s user 0.002s sys 0.032s
I20260806 22:39:16.488580 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36841
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:37945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:16.935300 8459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:16.935770 8459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:16.935973 8459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:16.944447 8459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:16.944658 8459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:16.944865 8459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:39:16.956632 8459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:37945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:16.958597 8459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:16.961056 8459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:16.979116 8464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:16.979059 8465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:16.979082 8467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:16.980098 8459 server_base.cc:1034] running on GCE node
I20260806 22:39:16.980985 8459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:16.982285 8459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:16.983547 8459 hybrid_clock.cc:648] HybridClock initialized: now 1786055956983474 us; error 66 us; skew 500 ppm
I20260806 22:39:16.984011 8459 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:16 dist-test-slave-h3ft krb5kdc[8318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055956, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:16.988049 8459 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:39:16.989941 8459 webserver.cc:460] Webserver started at http://127.15.198.129:45861/ using document root <none> and password file <none>
I20260806 22:39:16.990917 8459 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:16.991108 8459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:16.991534 8459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:16.995615 8459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad"
format_stamp: "Formatted at 2026-08-06 22:39:16 on dist-test-slave-h3ft"
server_key: "a8938359677b0a1c514a1a01bf37f623"
server_key_iv: "ceb4b468563e17f9a5fdf76061ed2afc"
server_key_version: "encryptionkey@0"
I20260806 22:39:16.996590 8459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad"
format_stamp: "Formatted at 2026-08-06 22:39:16 on dist-test-slave-h3ft"
server_key: "a8938359677b0a1c514a1a01bf37f623"
server_key_iv: "ceb4b468563e17f9a5fdf76061ed2afc"
server_key_version: "encryptionkey@0"
I20260806 22:39:17.003629 8459 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260806 22:39:17.008656 8480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:17.010617 8459 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:39:17.010877 8459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad"
format_stamp: "Formatted at 2026-08-06 22:39:16 on dist-test-slave-h3ft"
server_key: "a8938359677b0a1c514a1a01bf37f623"
server_key_iv: "ceb4b468563e17f9a5fdf76061ed2afc"
server_key_version: "encryptionkey@0"
I20260806 22:39:17.011227 8459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:17.033125 8459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:17.039038 8459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:17.039407 8459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:17.040733 8459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:17.042695 8459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:17.042794 8459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:17.042917 8459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:17.042970 8459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:17.085780 8459 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:39285
I20260806 22:39:17.085821 8634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:39285 every 8 connection(s)
I20260806 22:39:17.087719 8459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:39:17.097476 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 8459
I20260806 22:39:17.097810 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:39:17.098959 16154 external_mini_cluster.cc:1428] Setting key 82b9a9734d5120367b60302b951ddc09
Aug 06 22:39:17 dist-test-slave-h3ft krb5kdc[8318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055956, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:39:17.118287 8637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:17.091299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:60601 (local address 127.15.198.190:36841)
0806 22:39:17.091912 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:17.091919 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:17.093171 (+ 1252us) server_negotiation.cc:407] Connection header received
0806 22:39:17.095733 (+ 2562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:17.095741 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:17.095874 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:17.096034 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:17.098922 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:17.099926 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:17.102765 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:17.103575 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:17.103755 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:17.108958 (+ 5203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:17.108969 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:17.108977 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:17.109038 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:17.113456 (+ 4418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:17.114649 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:17.114659 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:17.114666 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:17.114892 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:17.115584 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:17.115592 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:17.115598 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:17.116095 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:17.116275 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:17.117350 (+ 1075us) server_negotiation.cc:299] Negotiation successful
0806 22:39:17.117727 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":339,"threads_started":1}
I20260806 22:39:17.119479 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:17.091841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:36841 (local address 127.15.198.129:60601)
0806 22:39:17.092896 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:17.092958 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:17.095347 (+ 2389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:17.096368 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:17.096381 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:17.097389 (+ 1008us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:17.098584 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:17.098625 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:17.100173 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:17.100183 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:17.102559 (+ 2376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:17.102580 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:17.103922 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:17.103929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:17.104264 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:17.105293 (+ 1029us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:17.105334 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:17.108597 (+ 3263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:17.113705 (+ 5108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:17.113717 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:17.113740 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:17.114458 (+ 718us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:17.115164 (+ 706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:17.115168 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:17.115171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:17.115441 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:17.116309 (+ 868us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:17.116317 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:17.116860 (+ 543us) client_negotiation.cc:769] Sending connection context
0806 22:39:17.117183 (+ 323us) client_negotiation.cc:240] Negotiation successful
0806 22:39:17.117852 (+ 669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":387,"threads_started":1}
I20260806 22:39:17.121862 8635 heartbeater.cc:347] Connected to a master server at 127.15.198.190:36841
I20260806 22:39:17.122413 8635 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:17.123751 8635 heartbeater.cc:511] Master 127.15.198.190:36841 requested a full tablet report, sending...
I20260806 22:39:17.127795 8376 ts_manager.cc:194] Registered new tserver with Master: 44b9cde44a8c4c8cad94a75c11d1f8ad (127.15.198.129:39285)
I20260806 22:39:17.130582 8376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:60601
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
I20260806 22:39:17.200399 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36841
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:37945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:17.662873 8642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:17.663367 8642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:17.663573 8642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:17.671823 8642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:17.672057 8642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:17.672276 8642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:39:17.684144 8642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:37945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:17.686165 8642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:17.688071 8642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:17.707377 8647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:17.707984 8652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:17.708150 8648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:17.713583 8642 server_base.cc:1034] running on GCE node
I20260806 22:39:17.714671 8642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:17.716631 8642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:17.717957 8642 hybrid_clock.cc:648] HybridClock initialized: now 1786055957717938 us; error 98 us; skew 500 ppm
I20260806 22:39:17.718492 8642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:17 dist-test-slave-h3ft krb5kdc[8318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055957, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:17.729540 8642 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:39:17.732263 8642 webserver.cc:460] Webserver started at http://127.15.198.130:45441/ using document root <none> and password file <none>
I20260806 22:39:17.733369 8642 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:17.733580 8642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:17.734110 8642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:17.738161 8642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "f30c44014f204ec7a25e082f752f4d77"
format_stamp: "Formatted at 2026-08-06 22:39:17 on dist-test-slave-h3ft"
server_key: "5d1b1130ba58c0d222b995cdd7e47838"
server_key_iv: "d03c1f72f7cc6904e940307335cdfda8"
server_key_version: "encryptionkey@0"
I20260806 22:39:17.739414 8642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "f30c44014f204ec7a25e082f752f4d77"
format_stamp: "Formatted at 2026-08-06 22:39:17 on dist-test-slave-h3ft"
server_key: "5d1b1130ba58c0d222b995cdd7e47838"
server_key_iv: "d03c1f72f7cc6904e940307335cdfda8"
server_key_version: "encryptionkey@0"
I20260806 22:39:17.746459 8642 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 22:39:17.751530 8659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:17.753609 8642 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260806 22:39:17.753896 8642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "f30c44014f204ec7a25e082f752f4d77"
format_stamp: "Formatted at 2026-08-06 22:39:17 on dist-test-slave-h3ft"
server_key: "5d1b1130ba58c0d222b995cdd7e47838"
server_key_iv: "d03c1f72f7cc6904e940307335cdfda8"
server_key_version: "encryptionkey@0"
I20260806 22:39:17.754222 8642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:17.783952 8642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:17.789446 8642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:17.789840 8642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:17.791242 8642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:17.793005 8642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:17.793120 8642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:17.793287 8642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:17.793376 8642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:17.836704 8642 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:44235
I20260806 22:39:17.836750 8816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:44235 every 8 connection(s)
I20260806 22:39:17.838450 8642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:39:17.852881 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 8642
I20260806 22:39:17.853165 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:39:17.853742 16154 external_mini_cluster.cc:1428] Setting key 77313b1a9072eaf80893bfe7fdce5212
Aug 06 22:39:17 dist-test-slave-h3ft krb5kdc[8318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055957, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:39:17.867156 8820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:17.842246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:52445 (local address 127.15.198.190:36841)
0806 22:39:17.842864 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:17.842870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:17.844198 (+ 1328us) server_negotiation.cc:407] Connection header received
0806 22:39:17.846665 (+ 2467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:17.846673 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:17.846795 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:17.846949 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:17.849271 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:17.850179 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:17.852620 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:17.853284 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:17.853468 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:17.859465 (+ 5997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:17.859473 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:17.859477 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:17.859522 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:17.863118 (+ 3596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:17.864105 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:17.864118 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:17.864121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:17.864286 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:17.864835 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:17.864839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:17.864841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:17.865205 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:17.865408 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:17.866319 (+ 911us) server_negotiation.cc:299] Negotiation successful
0806 22:39:17.866644 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":333,"threads_started":1}
I20260806 22:39:17.868562 8821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:17.842836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:36841 (local address 127.15.198.130:52445)
0806 22:39:17.843940 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:17.844013 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:17.846279 (+ 2266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:17.847302 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:17.847314 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:17.848120 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:17.849022 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:17.849046 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:17.850384 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:17.850388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:17.852343 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:17.852369 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:17.853631 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:17.853640 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:17.853996 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:17.855434 (+ 1438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:17.855474 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:17.859087 (+ 3613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:17.863303 (+ 4216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:17.863316 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:17.863340 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:17.863942 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:17.864422 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:17.864427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:17.864432 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:17.864680 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:17.865368 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:17.865379 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:17.865872 (+ 493us) client_negotiation.cc:769] Sending connection context
0806 22:39:17.866206 (+ 334us) client_negotiation.cc:240] Negotiation successful
0806 22:39:17.866957 (+ 751us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"thread_start_us":482,"threads_started":1}
I20260806 22:39:17.870781 8817 heartbeater.cc:347] Connected to a master server at 127.15.198.190:36841
I20260806 22:39:17.871309 8817 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:17.872610 8817 heartbeater.cc:511] Master 127.15.198.190:36841 requested a full tablet report, sending...
I20260806 22:39:17.874970 8376 ts_manager.cc:194] Registered new tserver with Master: f30c44014f204ec7a25e082f752f4d77 (127.15.198.130:44235)
I20260806 22:39:17.876246 8376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:52445
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
W20260806 22:39:17.967267 16154 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.198.131: real 0.109s user 0.002s sys 0.036s
I20260806 22:39:17.968328 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36841
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:37945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:39:18.137332 8635 heartbeater.cc:503] Master 127.15.198.190:36841 was elected leader, sending a full tablet report...
W20260806 22:39:18.421983 8827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:18.422467 8827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:18.422647 8827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:18.431196 8827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:18.431442 8827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:18.431666 8827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:39:18.443197 8827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:37945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:18.445148 8827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:18.446942 8827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:18.464025 8837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:18.464046 8834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:18.464087 8835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:18.465305 8827 server_base.cc:1034] running on GCE node
I20260806 22:39:18.466184 8827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:18.467573 8827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:18.468868 8827 hybrid_clock.cc:648] HybridClock initialized: now 1786055958468774 us; error 82 us; skew 500 ppm
I20260806 22:39:18.469334 8827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:18 dist-test-slave-h3ft krb5kdc[8318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055958, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:18.478113 8827 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:39:18.480379 8827 webserver.cc:460] Webserver started at http://127.15.198.131:44577/ using document root <none> and password file <none>
I20260806 22:39:18.481357 8827 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:18.481541 8827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:18.481972 8827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:18.486418 8827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "652743aae9ef462abeacc0c7a5ed410c"
format_stamp: "Formatted at 2026-08-06 22:39:18 on dist-test-slave-h3ft"
server_key: "264eddd836d2ebdb336f2f06ec4c6921"
server_key_iv: "d6c25477ca639070b1cc5b41ea8e1f3d"
server_key_version: "encryptionkey@0"
I20260806 22:39:18.487547 8827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "652743aae9ef462abeacc0c7a5ed410c"
format_stamp: "Formatted at 2026-08-06 22:39:18 on dist-test-slave-h3ft"
server_key: "264eddd836d2ebdb336f2f06ec4c6921"
server_key_iv: "d6c25477ca639070b1cc5b41ea8e1f3d"
server_key_version: "encryptionkey@0"
I20260806 22:39:18.494740 8827 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260806 22:39:18.499917 8845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:18.502075 8827 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 22:39:18.502374 8827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "652743aae9ef462abeacc0c7a5ed410c"
format_stamp: "Formatted at 2026-08-06 22:39:18 on dist-test-slave-h3ft"
server_key: "264eddd836d2ebdb336f2f06ec4c6921"
server_key_iv: "d6c25477ca639070b1cc5b41ea8e1f3d"
server_key_version: "encryptionkey@0"
I20260806 22:39:18.502691 8827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:18.530272 8827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:18.536463 8827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:18.536886 8827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:18.538237 8827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:18.540148 8827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:18.540244 8827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:18.540427 8827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:18.540483 8827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:18.583109 8827 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:38239
I20260806 22:39:18.583222 8999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:38239 every 8 connection(s)
I20260806 22:39:18.584952 8827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:39:18.585620 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 8827
I20260806 22:39:18.585856 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:39:18.586706 16154 external_mini_cluster.cc:1428] Setting key 0c64f7f21cf8c1f11945052cc666430b
Aug 06 22:39:18 dist-test-slave-h3ft krb5kdc[8318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055958, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:39:18.611248 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:18.588623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:38161 (local address 127.15.198.190:36841)
0806 22:39:18.589309 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:18.589316 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:18.591091 (+ 1775us) server_negotiation.cc:407] Connection header received
0806 22:39:18.593364 (+ 2273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:18.593369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:18.593466 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:18.593576 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:18.595593 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.596448 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.598240 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.598679 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.598786 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:18.603539 (+ 4753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:18.603546 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:18.603551 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:18.603597 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:18.606989 (+ 3392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:18.608156 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:18.608162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:18.608166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:18.608352 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:18.608934 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:18.608939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:18.608943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:18.609302 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:18.609501 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:18.610428 (+ 927us) server_negotiation.cc:299] Negotiation successful
0806 22:39:18.610742 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":283,"threads_started":1}
I20260806 22:39:18.612664 9005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:18.589574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:36841 (local address 127.15.198.131:38161)
0806 22:39:18.590695 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:18.590818 (+ 123us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:18.593065 (+ 2247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:18.593824 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:18.593834 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:18.594537 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:18.595401 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:18.595428 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.596638 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.596643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:18.598082 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:18.598096 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.598904 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.598908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:18.599270 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:18.600141 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:18.600170 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:18.603269 (+ 3099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:18.607243 (+ 3974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:18.607254 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:18.607297 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:18.607976 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:18.608502 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:18.608520 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:18.608522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:18.608789 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:18.609458 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:18.609468 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:18.609957 (+ 489us) client_negotiation.cc:769] Sending connection context
0806 22:39:18.610305 (+ 348us) client_negotiation.cc:240] Negotiation successful
0806 22:39:18.611045 (+ 740us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":473,"threads_started":1}
I20260806 22:39:18.614374 9000 heartbeater.cc:347] Connected to a master server at 127.15.198.190:36841
I20260806 22:39:18.614807 9000 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:18.615948 9000 heartbeater.cc:511] Master 127.15.198.190:36841 requested a full tablet report, sending...
I20260806 22:39:18.617983 8376 ts_manager.cc:194] Registered new tserver with Master: 652743aae9ef462abeacc0c7a5ed410c (127.15.198.131:38239)
I20260806 22:39:18.619108 8376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:38161
I20260806 22:39:18.630966 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:39:18.653736 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:18.637348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56210 (local address 127.15.198.190:36841)
0806 22:39:18.637585 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:18.637591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:18.638091 (+ 500us) server_negotiation.cc:407] Connection header received
0806 22:39:18.638277 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:18.638282 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:18.638375 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:18.638487 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:18.640388 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.641295 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.643148 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.643662 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.643788 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:18.646907 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:18.646914 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:18.646918 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:18.646983 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:18.650443 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:18.651434 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:18.651440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:18.651443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:18.651616 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:18.652131 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:18.652135 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:18.652137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:18.652556 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:18.652798 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:18.653209 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 22:39:18.653440 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 22:39:18.659870 8376 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56210:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:39:18.668668 8376 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:39:18.715169 9017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:18.685055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:39285 (local address 127.0.0.1:51990)
0806 22:39:18.697524 (+ 12469us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:18.697602 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:18.698030 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:18.700477 (+ 2447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:18.700490 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:18.700542 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:18.701291 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:18.701312 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.705042 (+ 3730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.705055 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:18.709345 (+ 4290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:18.709393 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.713286 (+ 3893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.713299 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:18.713754 (+ 455us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:18.713873 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 22:39:18.714090 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 22:39:18.714475 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11990,"thread_start_us":13480,"threads_started":1}
I20260806 22:39:18.717974 9018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:18.685375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51990 (local address 127.15.198.129:39285)
0806 22:39:18.696760 (+ 11385us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:18.696775 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:18.699362 (+ 2587us) server_negotiation.cc:407] Connection header received
0806 22:39:18.699571 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:18.699586 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:18.700053 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:18.701671 (+ 1618us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:18.702092 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.704746 (+ 2654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.709793 (+ 5047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.712728 (+ 2935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.716749 (+ 4021us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:18.717117 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:18.717259 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 22:39:18.717515 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11100,"thread_start_us":10860,"threads_started":1}
I20260806 22:39:18.723711 8547 tablet_service.cc:1467] Processing CreateTablet for tablet 367bf31d9e1045f7afadff36319f3171 (DEFAULT_TABLE table=test-table [id=ec94292c0bc44a0bbd61b5d877c1f759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:18.726452 8547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 367bf31d9e1045f7afadff36319f3171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:18.726665 9016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:18.685115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:44235 (local address 127.0.0.1:59052)
0806 22:39:18.698943 (+ 13828us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:18.698968 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:18.699157 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:18.717711 (+ 18554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:18.717717 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:18.717754 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:18.718243 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:18.718260 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.721031 (+ 2771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.721037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:18.723302 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:18.723323 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.725628 (+ 2305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.725635 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:18.725854 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:18.725904 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 22:39:18.726019 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 22:39:18.726196 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13629,"thread_start_us":11754,"threads_started":1}
I20260806 22:39:18.726718 9020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:18.685805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59052 (local address 127.15.198.130:44235)
0806 22:39:18.716687 (+ 30882us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:18.716696 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:18.716745 (+ 49us) server_negotiation.cc:407] Connection header received
0806 22:39:18.716909 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:18.716921 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:18.717402 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:18.717711 (+ 309us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:18.718632 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.720835 (+ 2203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.723516 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.725276 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.725564 (+ 288us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:18.725811 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:18.726096 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 22:39:18.726320 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30694,"thread_start_us":30501,"threads_started":1}
I20260806 22:39:18.728631 9017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:18.685544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:38239 (local address 127.0.0.1:56018)
0806 22:39:18.716863 (+ 31319us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:18.716889 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:18.717217 (+ 328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:18.718361 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:18.718366 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:18.718398 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:18.718672 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:18.718688 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.723218 (+ 4530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.723222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:18.725517 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:18.725538 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.727726 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.727733 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:18.727927 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:18.727965 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:39:18.728064 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:39:18.728236 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31132,"mutex_wait_us":3,"thread_start_us":30301,"threads_started":1}
I20260806 22:39:18.728772 9026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:18.686022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56018 (local address 127.15.198.131:38239)
0806 22:39:18.715526 (+ 29504us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:18.715543 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:18.717171 (+ 1628us) server_negotiation.cc:407] Connection header received
0806 22:39:18.717448 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:18.717460 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:18.717728 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:18.717968 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:18.718948 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.720848 (+ 1900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.725771 (+ 4923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.727517 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.727744 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:18.727953 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:18.728133 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 22:39:18.728336 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29123,"thread_start_us":25806,"threads_started":1}
I20260806 22:39:18.731849 8722 tablet_service.cc:1467] Processing CreateTablet for tablet 367bf31d9e1045f7afadff36319f3171 (DEFAULT_TABLE table=test-table [id=ec94292c0bc44a0bbd61b5d877c1f759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:18.733897 8722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 367bf31d9e1045f7afadff36319f3171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:18.733435 8909 tablet_service.cc:1467] Processing CreateTablet for tablet 367bf31d9e1045f7afadff36319f3171 (DEFAULT_TABLE table=test-table [id=ec94292c0bc44a0bbd61b5d877c1f759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:18.735199 8909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 367bf31d9e1045f7afadff36319f3171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:18.748296 9027 tablet_bootstrap.cc:492] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad: Bootstrap starting.
I20260806 22:39:18.752099 9028 tablet_bootstrap.cc:492] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c: Bootstrap starting.
I20260806 22:39:18.754228 9029 tablet_bootstrap.cc:492] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77: Bootstrap starting.
I20260806 22:39:18.754385 9027 tablet_bootstrap.cc:654] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:18.756021 9027 log.cc:824] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:18.757138 9028 tablet_bootstrap.cc:654] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:18.758627 9029 tablet_bootstrap.cc:654] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:18.758780 9028 log.cc:824] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:18.760139 9027 tablet_bootstrap.cc:492] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad: No bootstrap required, opened a new log
I20260806 22:39:18.760237 9029 log.cc:824] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:18.760519 9027 ts_tablet_manager.cc:1397] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260806 22:39:18.763134 9028 tablet_bootstrap.cc:492] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c: No bootstrap required, opened a new log
I20260806 22:39:18.763523 9028 ts_tablet_manager.cc:1397] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20260806 22:39:18.766284 9029 tablet_bootstrap.cc:492] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77: No bootstrap required, opened a new log
I20260806 22:39:18.766644 9029 ts_tablet_manager.cc:1397] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260806 22:39:18.773334 9027 raft_consensus.cc:348] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 39285 } } peers { permanent_uuid: "f30c44014f204ec7a25e082f752f4d77" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 44235 } } peers { permanent_uuid: "652743aae9ef462abeacc0c7a5ed410c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 38239 } }
I20260806 22:39:18.774070 9027 raft_consensus.cc:374] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:18.774199 9027 raft_consensus.cc:729] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44b9cde44a8c4c8cad94a75c11d1f8ad, State: Initialized, Role: FOLLOWER
I20260806 22:39:18.775138 9027 consensus_queue.cc:260] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 39285 } } peers { permanent_uuid: "f30c44014f204ec7a25e082f752f4d77" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 44235 } } peers { permanent_uuid: "652743aae9ef462abeacc0c7a5ed410c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 38239 } }
I20260806 22:39:18.775107 9029 raft_consensus.cc:348] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 39285 } } peers { permanent_uuid: "f30c44014f204ec7a25e082f752f4d77" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 44235 } } peers { permanent_uuid: "652743aae9ef462abeacc0c7a5ed410c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 38239 } }
I20260806 22:39:18.775657 9029 raft_consensus.cc:374] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:18.775763 9029 raft_consensus.cc:729] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30c44014f204ec7a25e082f752f4d77, State: Initialized, Role: FOLLOWER
I20260806 22:39:18.776026 9028 raft_consensus.cc:348] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 39285 } } peers { permanent_uuid: "f30c44014f204ec7a25e082f752f4d77" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 44235 } } peers { permanent_uuid: "652743aae9ef462abeacc0c7a5ed410c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 38239 } }
I20260806 22:39:18.776654 9028 raft_consensus.cc:374] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:18.776774 9028 raft_consensus.cc:729] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652743aae9ef462abeacc0c7a5ed410c, State: Initialized, Role: FOLLOWER
I20260806 22:39:18.776623 9029 consensus_queue.cc:260] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 39285 } } peers { permanent_uuid: "f30c44014f204ec7a25e082f752f4d77" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 44235 } } peers { permanent_uuid: "652743aae9ef462abeacc0c7a5ed410c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 38239 } }
I20260806 22:39:18.777518 9028 consensus_queue.cc:260] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 39285 } } peers { permanent_uuid: "f30c44014f204ec7a25e082f752f4d77" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 44235 } } peers { permanent_uuid: "652743aae9ef462abeacc0c7a5ed410c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 38239 } }
I20260806 22:39:18.779793 9027 ts_tablet_manager.cc:1428] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad: Time spent starting tablet: real 0.019s user 0.015s sys 0.004s
I20260806 22:39:18.782390 9028 ts_tablet_manager.cc:1428] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c: Time spent starting tablet: real 0.019s user 0.010s sys 0.008s
I20260806 22:39:18.783128 9000 heartbeater.cc:503] Master 127.15.198.190:36841 was elected leader, sending a full tablet report...
I20260806 22:39:18.783742 8817 heartbeater.cc:503] Master 127.15.198.190:36841 was elected leader, sending a full tablet report...
I20260806 22:39:18.785015 9029 ts_tablet_manager.cc:1428] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77: Time spent starting tablet: real 0.018s user 0.017s sys 0.001s
W20260806 22:39:18.837491 9002 tablet.cc:2369] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:39:18.841214 8636 tablet.cc:2369] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:39:18.841564 8818 tablet.cc:2369] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:39:18.960664 9036 raft_consensus.cc:482] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:39:18.961160 9036 raft_consensus.cc:504] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 39285 } } peers { permanent_uuid: "f30c44014f204ec7a25e082f752f4d77" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 44235 } } peers { permanent_uuid: "652743aae9ef462abeacc0c7a5ed410c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 38239 } }
I20260806 22:39:18.963583 9036 leader_election.cc:288] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f30c44014f204ec7a25e082f752f4d77 (127.15.198.130:44235), 652743aae9ef462abeacc0c7a5ed410c (127.15.198.131:38239)
I20260806 22:39:18.976215 9026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:18.964589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:33453 (local address 127.15.198.131:38239)
0806 22:39:18.964837 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:18.964846 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:18.965748 (+ 902us) server_negotiation.cc:407] Connection header received
0806 22:39:18.966026 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:18.966033 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:18.966146 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:18.966272 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:18.967639 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.968961 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.973011 (+ 4050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.974485 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.974706 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:18.974803 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:18.975683 (+ 880us) server_negotiation.cc:299] Negotiation successful
0806 22:39:18.975868 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 22:39:18.976215 9020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:18.964335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:38487 (local address 127.15.198.130:44235)
0806 22:39:18.964656 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:18.964663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:18.965740 (+ 1077us) server_negotiation.cc:407] Connection header received
0806 22:39:18.966026 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:18.966033 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:18.966141 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:18.966278 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:18.967635 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.968984 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.973014 (+ 4030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.974496 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.974706 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:18.974803 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:18.975683 (+ 880us) server_negotiation.cc:299] Negotiation successful
0806 22:39:18.975868 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260806 22:39:18.976574 9039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:18.964257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:38239 (local address 127.15.198.129:33453)
0806 22:39:18.965603 (+ 1346us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:18.965620 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:18.965828 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:18.966429 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:18.966436 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:18.966488 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:18.967324 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:18.967349 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.969240 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.969245 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:18.972713 (+ 3468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:18.972744 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.974919 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.974937 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:18.975397 (+ 460us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:18.975485 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 22:39:18.975634 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 22:39:18.975930 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1228,"mutex_wait_us":661,"spinlock_wait_cycles":34816,"thread_start_us":494,"threads_started":1}
I20260806 22:39:18.976922 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:18.964233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:44235 (local address 127.15.198.129:38487)
0806 22:39:18.965539 (+ 1306us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:18.965564 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:18.965828 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:18.966429 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:18.966436 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:18.966488 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:18.967354 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:18.967372 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.969238 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.969245 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:18.972713 (+ 3468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:18.972744 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:18.974951 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:18.974958 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:18.975381 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:18.975485 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 22:39:18.975634 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 22:39:18.975930 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1120,"mutex_wait_us":303,"spinlock_wait_cycles":40960,"thread_start_us":823,"threads_started":1}
I20260806 22:39:18.978298 8751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "367bf31d9e1045f7afadff36319f3171" candidate_uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30c44014f204ec7a25e082f752f4d77" is_pre_election: true
I20260806 22:39:18.978296 8943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "367bf31d9e1045f7afadff36319f3171" candidate_uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652743aae9ef462abeacc0c7a5ed410c" is_pre_election: true
I20260806 22:39:18.979148 8943 raft_consensus.cc:2454] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44b9cde44a8c4c8cad94a75c11d1f8ad in term 0.
I20260806 22:39:18.979148 8751 raft_consensus.cc:2454] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44b9cde44a8c4c8cad94a75c11d1f8ad in term 0.
I20260806 22:39:18.980247 8481 leader_election.cc:302] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44b9cde44a8c4c8cad94a75c11d1f8ad, f30c44014f204ec7a25e082f752f4d77; no voters:
I20260806 22:39:18.980906 9036 raft_consensus.cc:2781] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:39:18.981120 9036 raft_consensus.cc:482] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:39:18.981285 9036 raft_consensus.cc:3037] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:18.984540 9036 raft_consensus.cc:504] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 39285 } } peers { permanent_uuid: "f30c44014f204ec7a25e082f752f4d77" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 44235 } } peers { permanent_uuid: "652743aae9ef462abeacc0c7a5ed410c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 38239 } }
I20260806 22:39:18.985357 9036 leader_election.cc:288] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [CANDIDATE]: Term 1 election: Requested vote from peers f30c44014f204ec7a25e082f752f4d77 (127.15.198.130:44235), 652743aae9ef462abeacc0c7a5ed410c (127.15.198.131:38239)
I20260806 22:39:18.986110 8943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "367bf31d9e1045f7afadff36319f3171" candidate_uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652743aae9ef462abeacc0c7a5ed410c"
I20260806 22:39:18.986142 8751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "367bf31d9e1045f7afadff36319f3171" candidate_uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30c44014f204ec7a25e082f752f4d77"
I20260806 22:39:18.986402 8943 raft_consensus.cc:3037] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:18.986431 8751 raft_consensus.cc:3037] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:18.989194 8943 raft_consensus.cc:2454] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44b9cde44a8c4c8cad94a75c11d1f8ad in term 1.
I20260806 22:39:18.989198 8751 raft_consensus.cc:2454] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44b9cde44a8c4c8cad94a75c11d1f8ad in term 1.
I20260806 22:39:18.989892 8481 leader_election.cc:302] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 44b9cde44a8c4c8cad94a75c11d1f8ad, 652743aae9ef462abeacc0c7a5ed410c, f30c44014f204ec7a25e082f752f4d77; no voters:
I20260806 22:39:18.990319 9036 raft_consensus.cc:2781] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:18.990927 9036 raft_consensus.cc:686] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [term 1 LEADER]: Becoming Leader. State: Replica: 44b9cde44a8c4c8cad94a75c11d1f8ad, State: Running, Role: LEADER
I20260806 22:39:18.991675 9036 consensus_queue.cc:237] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 39285 } } peers { permanent_uuid: "f30c44014f204ec7a25e082f752f4d77" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 44235 } } peers { permanent_uuid: "652743aae9ef462abeacc0c7a5ed410c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 38239 } }
I20260806 22:39:18.997664 8376 catalog_manager.cc:5495] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 44b9cde44a8c4c8cad94a75c11d1f8ad (127.15.198.129). New cstate: current_term: 1 leader_uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44b9cde44a8c4c8cad94a75c11d1f8ad" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 39285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f30c44014f204ec7a25e082f752f4d77" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 44235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652743aae9ef462abeacc0c7a5ed410c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 38239 } health_report { overall_health: UNKNOWN } } }
I20260806 22:39:19.029810 9018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:19.023199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52002 (local address 127.15.198.129:39285)
0806 22:39:19.023483 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:19.023489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:19.023517 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:39:19.023669 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:19.023673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:19.023775 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:19.024012 (+ 237us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:39:19.024485 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:19.025347 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:19.027043 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:19.027523 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:19.027681 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:19.028355 (+ 674us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:39:19.029076 (+ 721us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:39:19.029284 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:19.029344 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 22:39:19.029469 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 22:39:19.041987 8943 raft_consensus.cc:1261] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c [term 1 FOLLOWER]: Refusing update from remote peer 44b9cde44a8c4c8cad94a75c11d1f8ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:19.041986 8751 raft_consensus.cc:1261] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77 [term 1 FOLLOWER]: Refusing update from remote peer 44b9cde44a8c4c8cad94a75c11d1f8ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:19.043295 9042 consensus_queue.cc:1035] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30c44014f204ec7a25e082f752f4d77" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 44235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:19.043797 9036 consensus_queue.cc:1035] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "652743aae9ef462abeacc0c7a5ed410c" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 38239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:19.061057 9049 mvcc.cc:205] Tried to move back new op lower bound from 7315685208221966336 to 7315685208034865152. Current Snapshot: MvccSnapshot[applied={T|T < 7315685208221966336}]
I20260806 22:39:19.077936 8376 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:39:19.078337 8376 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:39:19.083815 8376 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76ccf26159834bb6b4d6c6dca2ae5a56: Sending DeleteTablet for 3 replicas of tablet 367bf31d9e1045f7afadff36319f3171
I20260806 22:39:19.085915 8909 tablet_service.cc:1514] Processing DeleteTablet for tablet 367bf31d9e1045f7afadff36319f3171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:19 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:56018
I20260806 22:39:19.085915 8722 tablet_service.cc:1514] Processing DeleteTablet for tablet 367bf31d9e1045f7afadff36319f3171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:19 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:59052
I20260806 22:39:19.091440 8547 tablet_service.cc:1514] Processing DeleteTablet for tablet 367bf31d9e1045f7afadff36319f3171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:19 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:51990
I20260806 22:39:19.094897 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 8459
I20260806 22:39:19.111292 9066 tablet_replica.cc:330] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad: stopping tablet replica
I20260806 22:39:19.113802 9066 raft_consensus.cc:2229] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:39:19.115182 9066 raft_consensus.cc:2258] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:19.120164 9066 ts_tablet_manager.cc:1905] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:19.120803 9065 tablet_replica.cc:330] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c: stopping tablet replica
I20260806 22:39:19.121605 9065 raft_consensus.cc:2229] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:39:19.122505 9065 raft_consensus.cc:2258] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:19.123802 9064 tablet_replica.cc:330] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77: stopping tablet replica
Aug 06 22:39:19 dist-test-slave-h3ft krb5kdc[8318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:39:19.124413 9064 raft_consensus.cc:2229] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:39:19.131641 9064 raft_consensus.cc:2258] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:19.133154 9065 ts_tablet_manager.cc:1905] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:19.134111 9064 ts_tablet_manager.cc:1905] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:19.142026 9065 ts_tablet_manager.cc:1918] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:19.142372 9065 log.cc:1196] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/367bf31d9e1045f7afadff36319f3171
I20260806 22:39:19.143177 9065 ts_tablet_manager.cc:1939] T 367bf31d9e1045f7afadff36319f3171 P 652743aae9ef462abeacc0c7a5ed410c: Deleting consensus metadata
I20260806 22:39:19.144610 9064 ts_tablet_manager.cc:1918] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:19.144817 9064 log.cc:1196] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/367bf31d9e1045f7afadff36319f3171
I20260806 22:39:19.145426 9064 ts_tablet_manager.cc:1939] T 367bf31d9e1045f7afadff36319f3171 P f30c44014f204ec7a25e082f752f4d77: Deleting consensus metadata
I20260806 22:39:19.146108 8354 catalog_manager.cc:4841] TS 652743aae9ef462abeacc0c7a5ed410c (127.15.198.131:38239): tablet 367bf31d9e1045f7afadff36319f3171 (table test-table [id=ec94292c0bc44a0bbd61b5d877c1f759]) successfully deleted
I20260806 22:39:19.146394 9066 ts_tablet_manager.cc:1918] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:19.146620 9066 log.cc:1196] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/367bf31d9e1045f7afadff36319f3171
I20260806 22:39:19.147264 9066 ts_tablet_manager.cc:1939] T 367bf31d9e1045f7afadff36319f3171 P 44b9cde44a8c4c8cad94a75c11d1f8ad: Deleting consensus metadata
I20260806 22:39:19.147706 8352 catalog_manager.cc:4841] TS f30c44014f204ec7a25e082f752f4d77 (127.15.198.130:44235): tablet 367bf31d9e1045f7afadff36319f3171 (table test-table [id=ec94292c0bc44a0bbd61b5d877c1f759]) successfully deleted
I20260806 22:39:19.149150 8355 catalog_manager.cc:4841] TS 44b9cde44a8c4c8cad94a75c11d1f8ad (127.15.198.129:39285): tablet 367bf31d9e1045f7afadff36319f3171 (table test-table [id=ec94292c0bc44a0bbd61b5d877c1f759]) successfully deleted
I20260806 22:39:19.150535 9018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:19.097192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52018 (local address 127.15.198.129:39285)
0806 22:39:19.097607 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:19.097619 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:19.110382 (+ 12763us) server_negotiation.cc:407] Connection header received
0806 22:39:19.111282 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:19.111292 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:19.111449 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:19.111567 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:19.115391 (+ 3824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:19.117204 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:19.124551 (+ 7347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:19.125382 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:19.125538 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:19.137072 (+ 11534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:19.137091 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:19.137115 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:19.137168 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:19.142244 (+ 5076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:19.146957 (+ 4713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:19.146970 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:19.146974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:19.147283 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:19.148024 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:19.148030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:19.148036 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:19.148668 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:19.149057 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:19.149770 (+ 713us) server_negotiation.cc:299] Negotiation successful
0806 22:39:19.150105 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260806 22:39:19.151532 8627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:19.267385 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 8459
I20260806 22:39:19.302186 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 8642
Aug 06 22:39:19 dist-test-slave-h3ft krb5kdc[8318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.130@KRBTEST.COM
I20260806 22:39:19.320436 9020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:19.303263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59066 (local address 127.15.198.130:44235)
0806 22:39:19.303518 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:19.303525 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:19.303562 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:39:19.303747 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:19.303753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:19.303836 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:19.303972 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:19.305497 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:19.306305 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:19.307919 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:19.308431 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:19.308603 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:19.313634 (+ 5031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:19.313658 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:19.313680 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:19.313728 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:19.317072 (+ 3344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:19.317875 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:19.317884 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:19.317887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:19.318047 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:19.318475 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:19.318479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:19.318481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:19.318969 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:19.319401 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:19.319722 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 22:39:19.319987 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 22:39:19.321375 8812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:19.422636 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 8642
I20260806 22:39:19.451282 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 8827
Aug 06 22:39:19 dist-test-slave-h3ft krb5kdc[8318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.131@KRBTEST.COM
I20260806 22:39:19.471848 9026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:19.452327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56034 (local address 127.15.198.131:38239)
0806 22:39:19.452616 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:19.452623 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:19.452655 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:39:19.452984 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:19.452994 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:19.453091 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:19.453223 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:19.454970 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:19.456067 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:19.457626 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:19.458105 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:19.458261 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:19.464293 (+ 6032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:19.464318 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:19.464349 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:19.464405 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:19.467994 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:19.468963 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:19.468975 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:19.468978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:19.469180 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:19.469662 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:19.469668 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:19.469677 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:19.470194 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:19.470609 (+ 415us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:19.470946 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 22:39:19.471288 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 22:39:19.472739 8995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:19.574267 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 8827
I20260806 22:39:19.604068 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 8335
I20260806 22:39:19.605569 8417 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:19.738588 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 8335
2026-08-06T22:39:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5959 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T22:39:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:39:19Z Disabled control of system clock
I20260806 22:39:19.867756 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:41277
--webserver_interface=127.15.198.190
--webserver_port=0
--builtin_ntp_servers=127.15.198.148:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:41277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:39:20.335754 9077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:20.336158 9077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:20.336313 9077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:20.344931 9077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:20.345103 9077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:20.345175 9077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:20.345273 9077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:20.345376 9077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:20.357273 9077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:41277
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:41277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:20.359349 9077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:20.361831 9077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:20.380354 9084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:20.380436 9083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:20.380440 9086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:20.381458 9077 server_base.cc:1034] running on GCE node
I20260806 22:39:20.382750 9077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:20.384869 9077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:20.386160 9077 hybrid_clock.cc:648] HybridClock initialized: now 1786055960386105 us; error 55 us; skew 500 ppm
I20260806 22:39:20.386636 9077 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:39:20.389900 9077 webserver.cc:460] Webserver started at http://127.15.198.190:35889/ using document root <none> and password file <none>
I20260806 22:39:20.390836 9077 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:20.391044 9077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:20.391480 9077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:20.395241 9077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "099d0a55d7004612940bd693e18fd1bf"
format_stamp: "Formatted at 2026-08-06 22:39:20 on dist-test-slave-h3ft"
server_key: "b096fbc9a4c5a3fd839e8928cb92a3e3"
server_key_iv: "3d06d2932b905d5047ce0fb32df73ef5"
server_key_version: "encryptionkey@0"
I20260806 22:39:20.396256 9077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "099d0a55d7004612940bd693e18fd1bf"
format_stamp: "Formatted at 2026-08-06 22:39:20 on dist-test-slave-h3ft"
server_key: "b096fbc9a4c5a3fd839e8928cb92a3e3"
server_key_iv: "3d06d2932b905d5047ce0fb32df73ef5"
server_key_version: "encryptionkey@0"
I20260806 22:39:20.403601 9077 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 22:39:20.408835 9095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:20.410743 9077 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:39:20.411031 9077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "099d0a55d7004612940bd693e18fd1bf"
format_stamp: "Formatted at 2026-08-06 22:39:20 on dist-test-slave-h3ft"
server_key: "b096fbc9a4c5a3fd839e8928cb92a3e3"
server_key_iv: "3d06d2932b905d5047ce0fb32df73ef5"
server_key_version: "encryptionkey@0"
I20260806 22:39:20.411338 9077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:20.454281 9077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:20.455615 9077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:20.456053 9077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:20.479372 9077 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:41277
I20260806 22:39:20.479398 9162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:41277 every 8 connection(s)
I20260806 22:39:20.481405 9077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:39:20.485512 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 9077
I20260806 22:39:20.485759 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:39:20.486331 16154 external_mini_cluster.cc:1428] Setting key 9abcd1e38eef89d7a9b4a302e1b889c9
I20260806 22:39:20.486871 9163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:20.504904 9163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf: Bootstrap starting.
I20260806 22:39:20.505081 9166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:20.488964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56486 (local address 127.15.198.190:41277)
0806 22:39:20.490242 (+ 1278us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:20.490256 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:20.490314 (+ 58us) server_negotiation.cc:407] Connection header received
0806 22:39:20.492801 (+ 2487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:20.492836 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:20.493560 (+ 724us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:20.494063 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:20.495166 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:20.496770 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:20.499081 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:20.499943 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:20.500154 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:20.501881 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:20.501890 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:20.501911 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:20.502677 (+ 766us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:20.502809 (+ 132us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:20.502835 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:20.502932 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:20.503266 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 22:39:20.503748 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":472,"threads_started":1}
I20260806 22:39:20.509347 9163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:20.510707 9163 log.cc:824] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:20.514891 9163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf: No bootstrap required, opened a new log
I20260806 22:39:20.524178 9163 raft_consensus.cc:348] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099d0a55d7004612940bd693e18fd1bf" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41277 } }
I20260806 22:39:20.524709 9163 raft_consensus.cc:374] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:20.524822 9163 raft_consensus.cc:729] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 099d0a55d7004612940bd693e18fd1bf, State: Initialized, Role: FOLLOWER
I20260806 22:39:20.525650 9163 consensus_queue.cc:260] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099d0a55d7004612940bd693e18fd1bf" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41277 } }
I20260806 22:39:20.525972 9163 raft_consensus.cc:388] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:39:20.526168 9163 raft_consensus.cc:482] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:39:20.526381 9163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:20.529172 9163 raft_consensus.cc:504] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099d0a55d7004612940bd693e18fd1bf" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41277 } }
I20260806 22:39:20.529881 9163 leader_election.cc:302] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 099d0a55d7004612940bd693e18fd1bf; no voters:
I20260806 22:39:20.530712 9163 leader_election.cc:288] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:39:20.530956 9168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:20.531958 9168 raft_consensus.cc:686] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [term 1 LEADER]: Becoming Leader. State: Replica: 099d0a55d7004612940bd693e18fd1bf, State: Running, Role: LEADER
I20260806 22:39:20.532614 9168 consensus_queue.cc:237] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099d0a55d7004612940bd693e18fd1bf" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41277 } }
I20260806 22:39:20.533624 9163 sys_catalog.cc:564] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:20.535979 9170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 099d0a55d7004612940bd693e18fd1bf. Latest consensus state: current_term: 1 leader_uuid: "099d0a55d7004612940bd693e18fd1bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099d0a55d7004612940bd693e18fd1bf" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41277 } } }
I20260806 22:39:20.536341 9169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "099d0a55d7004612940bd693e18fd1bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099d0a55d7004612940bd693e18fd1bf" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 41277 } } }
I20260806 22:39:20.536896 9169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:20.536892 9170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:20.538625 9173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:39:20.547824 9173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:39:20.560045 9173 catalog_manager.cc:1347] Generated new cluster ID: 1de3a8c61387406388ea298849322b88
I20260806 22:39:20.560237 9173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:39:20.582363 9173 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:39:20.584455 9173 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:39:20.611137 9173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf: Generated new TSK 0
I20260806 22:39:20.612771 9173 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:39:20.623558 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:41277
--builtin_ntp_servers=127.15.198.148:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:39:21.092581 9193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:21.093035 9193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:21.093216 9193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:21.102736 9193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:21.102912 9193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:21.103147 9193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:39:21.114030 9193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:41277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:21.116073 9193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:21.118553 9193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:21.136901 9199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:21.136894 9200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:21.136942 9203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:21.138083 9193 server_base.cc:1034] running on GCE node
I20260806 22:39:21.138886 9193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:21.140302 9193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:21.141582 9193 hybrid_clock.cc:648] HybridClock initialized: now 1786055961141505 us; error 81 us; skew 500 ppm
I20260806 22:39:21.142071 9193 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:39:21.145200 9193 webserver.cc:460] Webserver started at http://127.15.198.129:37265/ using document root <none> and password file <none>
I20260806 22:39:21.146205 9193 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:21.146377 9193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:21.146806 9193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:21.150784 9193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "a8d2695ad18844638924852fe7743ed1"
format_stamp: "Formatted at 2026-08-06 22:39:21 on dist-test-slave-h3ft"
server_key: "e1e8159bb24761a46029e14e557ba43c"
server_key_iv: "c36d682c6860b790e91c245f1307fb50"
server_key_version: "encryptionkey@0"
I20260806 22:39:21.151850 9193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "a8d2695ad18844638924852fe7743ed1"
format_stamp: "Formatted at 2026-08-06 22:39:21 on dist-test-slave-h3ft"
server_key: "e1e8159bb24761a46029e14e557ba43c"
server_key_iv: "c36d682c6860b790e91c245f1307fb50"
server_key_version: "encryptionkey@0"
I20260806 22:39:21.158581 9193 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 22:39:21.163373 9213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:21.165145 9193 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:39:21.165366 9193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "a8d2695ad18844638924852fe7743ed1"
format_stamp: "Formatted at 2026-08-06 22:39:21 on dist-test-slave-h3ft"
server_key: "e1e8159bb24761a46029e14e557ba43c"
server_key_iv: "c36d682c6860b790e91c245f1307fb50"
server_key_version: "encryptionkey@0"
I20260806 22:39:21.165644 9193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:21.187649 9193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:21.188841 9193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:21.189256 9193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:21.190647 9193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:21.192746 9193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:21.192899 9193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:21.193089 9193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:21.193192 9193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:21.233286 9193 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:34617
I20260806 22:39:21.233341 9369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:34617 every 8 connection(s)
I20260806 22:39:21.235224 9193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:39:21.242535 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 9193
I20260806 22:39:21.242759 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:39:21.243503 16154 external_mini_cluster.cc:1428] Setting key cbc23fb1986d4b8e4a03cb647f518e16
I20260806 22:39:21.247556 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:41277
--builtin_ntp_servers=127.15.198.148:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:39:21.252955 9373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:21.238481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:44697 (local address 127.15.198.190:41277)
0806 22:39:21.239143 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:21.239150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:21.240634 (+ 1484us) server_negotiation.cc:407] Connection header received
0806 22:39:21.243034 (+ 2400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:21.243042 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:21.243175 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:21.243328 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:21.245165 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:21.246289 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:21.248676 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:21.249521 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:21.249720 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:21.251712 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:21.251722 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:21.251730 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:21.251766 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:21.251840 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:21.251861 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:21.251974 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:21.252273 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 22:39:21.252437 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":294,"threads_started":1}
I20260806 22:39:21.254240 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:21.239308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41277 (local address 127.15.198.129:44697)
0806 22:39:21.240452 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:21.240497 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:21.242536 (+ 2039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:21.243756 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:21.243768 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:21.243937 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:21.244946 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:21.244972 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:21.246498 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:21.246504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:21.248453 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:21.248475 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:21.249855 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:21.249861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:21.250205 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:21.251215 (+ 1010us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:39:21.251218 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:21.251286 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:39:21.251425 (+ 139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:21.251976 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:21.251985 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:21.251993 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:39:21.252205 (+ 212us) client_negotiation.cc:240] Negotiation successful
0806 22:39:21.252673 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":319,"threads_started":1}
I20260806 22:39:21.256392 9370 heartbeater.cc:347] Connected to a master server at 127.15.198.190:41277
I20260806 22:39:21.256875 9370 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:21.258216 9370 heartbeater.cc:511] Master 127.15.198.190:41277 requested a full tablet report, sending...
I20260806 22:39:21.261435 9117 ts_manager.cc:194] Registered new tserver with Master: a8d2695ad18844638924852fe7743ed1 (127.15.198.129:34617)
I20260806 22:39:21.263435 9117 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.198.129:44697
W20260806 22:39:21.679656 9375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:21.680081 9375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:21.680188 9375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:21.688668 9375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:21.688843 9375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:21.688974 9375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:39:21.700325 9375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:41277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:21.702512 9375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:21.705147 9375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:21.724335 9381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:21.724287 9383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:21.724299 9387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:21.725943 9375 server_base.cc:1034] running on GCE node
I20260806 22:39:21.726881 9375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:21.728297 9375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:21.729592 9375 hybrid_clock.cc:648] HybridClock initialized: now 1786055961729526 us; error 62 us; skew 500 ppm
I20260806 22:39:21.730057 9375 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:39:21.733429 9375 webserver.cc:460] Webserver started at http://127.15.198.130:43851/ using document root <none> and password file <none>
I20260806 22:39:21.734555 9375 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:21.734720 9375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:21.735220 9375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:21.739233 9375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "f643f074dbad48378878d182cc9f7373"
format_stamp: "Formatted at 2026-08-06 22:39:21 on dist-test-slave-h3ft"
server_key: "c8c4d89ba3485e1647b2536f1cc64721"
server_key_iv: "3bd17adb4ef9f6d49999d94a32feaee7"
server_key_version: "encryptionkey@0"
I20260806 22:39:21.740334 9375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "f643f074dbad48378878d182cc9f7373"
format_stamp: "Formatted at 2026-08-06 22:39:21 on dist-test-slave-h3ft"
server_key: "c8c4d89ba3485e1647b2536f1cc64721"
server_key_iv: "3bd17adb4ef9f6d49999d94a32feaee7"
server_key_version: "encryptionkey@0"
I20260806 22:39:21.748031 9375 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260806 22:39:21.752673 9394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:21.754927 9375 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260806 22:39:21.755242 9375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "f643f074dbad48378878d182cc9f7373"
format_stamp: "Formatted at 2026-08-06 22:39:21 on dist-test-slave-h3ft"
server_key: "c8c4d89ba3485e1647b2536f1cc64721"
server_key_iv: "3bd17adb4ef9f6d49999d94a32feaee7"
server_key_version: "encryptionkey@0"
I20260806 22:39:21.755558 9375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:21.774302 9375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:21.775614 9375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:21.776087 9375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:21.777354 9375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:21.779295 9375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:21.779403 9375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:21.779493 9375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:21.779553 9375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:21.819749 9375 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:41075
I20260806 22:39:21.819811 9551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:41075 every 8 connection(s)
I20260806 22:39:21.821480 9375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:39:21.825333 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 9375
I20260806 22:39:21.825583 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:39:21.826180 16154 external_mini_cluster.cc:1428] Setting key e2eef2b18962743c6d98794536ec6d0b
I20260806 22:39:21.830354 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:41277
--builtin_ntp_servers=127.15.198.148:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:39:21.839444 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:21.824870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:47563 (local address 127.15.198.190:41277)
0806 22:39:21.825559 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:21.825569 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:21.827451 (+ 1882us) server_negotiation.cc:407] Connection header received
0806 22:39:21.829911 (+ 2460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:21.829919 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:21.830037 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:21.830194 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:21.831731 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:21.833076 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:21.835299 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:21.836101 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:21.836268 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:21.838191 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:21.838201 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:21.838208 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:21.838245 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:21.838318 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:21.838338 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:21.838441 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:21.838731 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 22:39:21.838890 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":303,"threads_started":1}
I20260806 22:39:21.840770 9557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:21.825862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41277 (local address 127.15.198.130:47563)
0806 22:39:21.827229 (+ 1367us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:21.827302 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:21.829536 (+ 2234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:21.830460 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:21.830472 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:21.830649 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:21.831510 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:21.831534 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:21.833312 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:21.833317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:21.835093 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:21.835120 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:21.836425 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:21.836432 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:21.836703 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:21.837692 (+ 989us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:39:21.837694 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:21.837757 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:39:21.837879 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:21.838457 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:21.838467 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:21.838475 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:39:21.838660 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 22:39:21.839186 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":416,"threads_started":1}
I20260806 22:39:21.842772 9552 heartbeater.cc:347] Connected to a master server at 127.15.198.190:41277
I20260806 22:39:21.843309 9552 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:21.844343 9552 heartbeater.cc:511] Master 127.15.198.190:41277 requested a full tablet report, sending...
I20260806 22:39:21.846302 9117 ts_manager.cc:194] Registered new tserver with Master: f643f074dbad48378878d182cc9f7373 (127.15.198.130:41075)
I20260806 22:39:21.847445 9117 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.198.130:47563
I20260806 22:39:22.267732 9370 heartbeater.cc:503] Master 127.15.198.190:41277 was elected leader, sending a full tablet report...
W20260806 22:39:22.279630 9559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:22.280107 9559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:22.280289 9559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:22.289407 9559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:22.289644 9559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:22.289856 9559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:39:22.300951 9559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:41277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:22.302882 9559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:22.305429 9559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:22.322803 9568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:22.322765 9570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:22.322741 9572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:22.324283 9559 server_base.cc:1034] running on GCE node
I20260806 22:39:22.325083 9559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:22.326455 9559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:22.327742 9559 hybrid_clock.cc:648] HybridClock initialized: now 1786055962327675 us; error 57 us; skew 500 ppm
I20260806 22:39:22.328184 9559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:39:22.331313 9559 webserver.cc:460] Webserver started at http://127.15.198.131:34067/ using document root <none> and password file <none>
I20260806 22:39:22.332422 9559 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:22.332682 9559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:22.333201 9559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:22.337247 9559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "33bc277c32134140b7ec5fd05478a998"
format_stamp: "Formatted at 2026-08-06 22:39:22 on dist-test-slave-h3ft"
server_key: "2bcb4ce749dfc6ce8df829df9071f516"
server_key_iv: "aaaa1d9a53bf49c036daaaba3bb7646c"
server_key_version: "encryptionkey@0"
I20260806 22:39:22.338241 9559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "33bc277c32134140b7ec5fd05478a998"
format_stamp: "Formatted at 2026-08-06 22:39:22 on dist-test-slave-h3ft"
server_key: "2bcb4ce749dfc6ce8df829df9071f516"
server_key_iv: "aaaa1d9a53bf49c036daaaba3bb7646c"
server_key_version: "encryptionkey@0"
I20260806 22:39:22.345269 9559 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 22:39:22.349691 9581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:22.351454 9559 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 22:39:22.351667 9559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "33bc277c32134140b7ec5fd05478a998"
format_stamp: "Formatted at 2026-08-06 22:39:22 on dist-test-slave-h3ft"
server_key: "2bcb4ce749dfc6ce8df829df9071f516"
server_key_iv: "aaaa1d9a53bf49c036daaaba3bb7646c"
server_key_version: "encryptionkey@0"
I20260806 22:39:22.351950 9559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:22.391472 9559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:22.392709 9559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:22.393164 9559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:22.394367 9559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:22.396313 9559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:22.396450 9559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:22.396605 9559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:22.396703 9559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:22.436491 9559 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:40365
I20260806 22:39:22.436578 9737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:40365 every 8 connection(s)
I20260806 22:39:22.438232 9559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:39:22.439505 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 9559
I20260806 22:39:22.439735 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:39:22.440454 16154 external_mini_cluster.cc:1428] Setting key 01e166cd63f5ece4a7d203f5ba5bdf3c
I20260806 22:39:22.454830 9743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:22.441975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:60379 (local address 127.15.198.190:41277)
0806 22:39:22.442607 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:22.442614 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:22.444707 (+ 2093us) server_negotiation.cc:407] Connection header received
0806 22:39:22.446865 (+ 2158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:22.446871 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:22.446994 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:22.447183 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:22.448561 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.449424 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.451258 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.451735 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.451893 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:22.453719 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:22.453725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:22.453728 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:22.453754 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:22.453816 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:22.453830 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:22.454002 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:22.454223 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 22:39:22.454362 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":324,"threads_started":1}
I20260806 22:39:22.456559 9744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:22.443075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:41277 (local address 127.15.198.131:60379)
0806 22:39:22.444516 (+ 1441us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:22.444578 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:22.446539 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:22.447490 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:22.447501 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:22.447616 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:22.448353 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:22.448373 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.449614 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.449618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:22.451083 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:22.451115 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.452013 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.452022 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:22.452328 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:22.453256 (+ 928us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:39:22.453259 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:22.453315 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:39:22.453431 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:22.453938 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:22.453951 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:22.453961 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 22:39:22.454212 (+ 251us) client_negotiation.cc:240] Negotiation successful
0806 22:39:22.454775 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":402,"threads_started":1}
I20260806 22:39:22.458504 9738 heartbeater.cc:347] Connected to a master server at 127.15.198.190:41277
I20260806 22:39:22.458869 9738 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:22.459973 9738 heartbeater.cc:511] Master 127.15.198.190:41277 requested a full tablet report, sending...
I20260806 22:39:22.461884 9117 ts_manager.cc:194] Registered new tserver with Master: 33bc277c32134140b7ec5fd05478a998 (127.15.198.131:40365)
I20260806 22:39:22.462944 9117 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.198.131:60379
I20260806 22:39:22.472388 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:39:22.485702 9743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:22.478064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56494 (local address 127.15.198.190:41277)
0806 22:39:22.478394 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:22.478403 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:22.478798 (+ 395us) server_negotiation.cc:407] Connection header received
0806 22:39:22.479120 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:22.479125 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:22.479209 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:22.479351 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:22.479997 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.480823 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.482418 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.482884 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.483076 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:22.484515 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:22.484521 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:22.484524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:22.484549 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:22.484595 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:22.484609 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:22.484715 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:22.484911 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 22:39:22.485094 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260806 22:39:22.491456 9117 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:39:22.499989 9117 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:39:22.530843 9758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:22.518497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:34617 (local address 127.0.0.1:50546)
0806 22:39:22.522226 (+ 3729us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:22.522256 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:22.522386 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:22.523536 (+ 1150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:22.523541 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:22.523562 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:22.524050 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:22.524063 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.525578 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.525584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:22.528049 (+ 2465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:22.528062 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.529650 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.529657 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:22.529895 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:22.529956 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 22:39:22.530142 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 22:39:22.530380 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3599,"mutex_wait_us":1,"thread_start_us":2702,"threads_started":1}
I20260806 22:39:22.531661 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:22.517659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:41075 (local address 127.0.0.1:59370)
0806 22:39:22.519130 (+ 1471us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:22.519199 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:22.519463 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:22.524264 (+ 4801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:22.524271 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:22.524304 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:22.524728 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:22.524742 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.527074 (+ 2332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.527080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:22.528664 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:22.528678 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.530694 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.530703 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:22.530898 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:22.530948 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 22:39:22.531078 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 22:39:22.531233 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1001,"mutex_wait_us":20,"thread_start_us":1572,"threads_started":1}
I20260806 22:39:22.532337 9763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:22.519343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50546 (local address 127.15.198.129:34617)
0806 22:39:22.521035 (+ 1692us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:22.521043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:22.522533 (+ 1490us) server_negotiation.cc:407] Connection header received
0806 22:39:22.522685 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:22.522693 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:22.522939 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:22.523205 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:22.524252 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.525427 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.528269 (+ 2842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.529409 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.529674 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:22.529981 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:22.530209 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 22:39:22.530429 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1533,"thread_start_us":1360,"threads_started":1}
I20260806 22:39:22.533581 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:22.518400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:40365 (local address 127.0.0.1:36814)
0806 22:39:22.521452 (+ 3052us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:22.521474 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:22.521655 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:22.522921 (+ 1266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:22.522929 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:22.522961 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:22.523896 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:22.523918 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.526112 (+ 2194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.526116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:22.530132 (+ 4016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:22.530148 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.532731 (+ 2583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.532739 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:22.532932 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:22.532963 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 22:39:22.533045 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:39:22.533186 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2942,"mutex_wait_us":65,"thread_start_us":3549,"threads_started":1}
I20260806 22:39:22.533716 9757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:22.517965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59370 (local address 127.15.198.130:41075)
0806 22:39:22.523685 (+ 5720us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:22.523692 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:22.523720 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:39:22.523812 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:22.523816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:22.523991 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:22.524977 (+ 986us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:22.525230 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.526840 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.528889 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.530455 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.532119 (+ 1664us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:22.532402 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:22.532485 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 22:39:22.533222 (+ 737us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5520,"thread_start_us":5403,"threads_started":1}
I20260806 22:39:22.534549 9762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:22.518904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36814 (local address 127.15.198.131:40365)
0806 22:39:22.520278 (+ 1374us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:22.520286 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:22.521610 (+ 1324us) server_negotiation.cc:407] Connection header received
0806 22:39:22.521876 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:22.521888 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:22.522282 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:22.522543 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:22.524161 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.525935 (+ 1774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.530412 (+ 4477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.532128 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.532342 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:22.532583 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:22.533903 (+ 1320us) server_negotiation.cc:299] Negotiation successful
0806 22:39:22.534120 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1129,"spinlock_wait_cycles":22912,"thread_start_us":1098,"threads_started":1}
I20260806 22:39:22.537667 9462 tablet_service.cc:1467] Processing CreateTablet for tablet 9b64444b8e06472abf0455fe476ccb81 (DEFAULT_TABLE table=test-table [id=796c843e4fe14c54abe3dbf57845ff74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:22.537667 9277 tablet_service.cc:1467] Processing CreateTablet for tablet 9b64444b8e06472abf0455fe476ccb81 (DEFAULT_TABLE table=test-table [id=796c843e4fe14c54abe3dbf57845ff74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:22.538178 9650 tablet_service.cc:1467] Processing CreateTablet for tablet 9b64444b8e06472abf0455fe476ccb81 (DEFAULT_TABLE table=test-table [id=796c843e4fe14c54abe3dbf57845ff74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:22.539561 9462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b64444b8e06472abf0455fe476ccb81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:22.539564 9277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b64444b8e06472abf0455fe476ccb81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:22.539582 9650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b64444b8e06472abf0455fe476ccb81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:22.554184 9764 tablet_bootstrap.cc:492] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998: Bootstrap starting.
I20260806 22:39:22.556632 9765 tablet_bootstrap.cc:492] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1: Bootstrap starting.
I20260806 22:39:22.560281 9764 tablet_bootstrap.cc:654] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:22.560874 9767 tablet_bootstrap.cc:492] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373: Bootstrap starting.
I20260806 22:39:22.562240 9765 tablet_bootstrap.cc:654] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:22.562886 9764 log.cc:824] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:22.563532 9765 log.cc:824] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:22.566006 9767 tablet_bootstrap.cc:654] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:22.567229 9764 tablet_bootstrap.cc:492] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998: No bootstrap required, opened a new log
I20260806 22:39:22.567231 9765 tablet_bootstrap.cc:492] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1: No bootstrap required, opened a new log
I20260806 22:39:22.567337 9767 log.cc:824] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:22.567601 9764 ts_tablet_manager.cc:1397] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260806 22:39:22.567601 9765 ts_tablet_manager.cc:1397] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260806 22:39:22.570884 9767 tablet_bootstrap.cc:492] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373: No bootstrap required, opened a new log
I20260806 22:39:22.571388 9767 ts_tablet_manager.cc:1397] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260806 22:39:22.580397 9767 raft_consensus.cc:348] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f643f074dbad48378878d182cc9f7373" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 41075 } } peers { permanent_uuid: "a8d2695ad18844638924852fe7743ed1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 34617 } } peers { permanent_uuid: "33bc277c32134140b7ec5fd05478a998" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 40365 } }
I20260806 22:39:22.580286 9765 raft_consensus.cc:348] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f643f074dbad48378878d182cc9f7373" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 41075 } } peers { permanent_uuid: "a8d2695ad18844638924852fe7743ed1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 34617 } } peers { permanent_uuid: "33bc277c32134140b7ec5fd05478a998" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 40365 } }
I20260806 22:39:22.580286 9764 raft_consensus.cc:348] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f643f074dbad48378878d182cc9f7373" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 41075 } } peers { permanent_uuid: "a8d2695ad18844638924852fe7743ed1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 34617 } } peers { permanent_uuid: "33bc277c32134140b7ec5fd05478a998" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 40365 } }
I20260806 22:39:22.580904 9765 raft_consensus.cc:374] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:22.580917 9764 raft_consensus.cc:374] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:22.580901 9767 raft_consensus.cc:374] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:22.581027 9767 raft_consensus.cc:729] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f643f074dbad48378878d182cc9f7373, State: Initialized, Role: FOLLOWER
I20260806 22:39:22.581038 9765 raft_consensus.cc:729] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8d2695ad18844638924852fe7743ed1, State: Initialized, Role: FOLLOWER
I20260806 22:39:22.581032 9764 raft_consensus.cc:729] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33bc277c32134140b7ec5fd05478a998, State: Initialized, Role: FOLLOWER
I20260806 22:39:22.581830 9767 consensus_queue.cc:260] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f643f074dbad48378878d182cc9f7373" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 41075 } } peers { permanent_uuid: "a8d2695ad18844638924852fe7743ed1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 34617 } } peers { permanent_uuid: "33bc277c32134140b7ec5fd05478a998" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 40365 } }
I20260806 22:39:22.581840 9765 consensus_queue.cc:260] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f643f074dbad48378878d182cc9f7373" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 41075 } } peers { permanent_uuid: "a8d2695ad18844638924852fe7743ed1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 34617 } } peers { permanent_uuid: "33bc277c32134140b7ec5fd05478a998" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 40365 } }
I20260806 22:39:22.581838 9764 consensus_queue.cc:260] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f643f074dbad48378878d182cc9f7373" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 41075 } } peers { permanent_uuid: "a8d2695ad18844638924852fe7743ed1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 34617 } } peers { permanent_uuid: "33bc277c32134140b7ec5fd05478a998" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 40365 } }
I20260806 22:39:22.584209 9738 heartbeater.cc:503] Master 127.15.198.190:41277 was elected leader, sending a full tablet report...
I20260806 22:39:22.584496 9765 ts_tablet_manager.cc:1428] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1: Time spent starting tablet: real 0.017s user 0.015s sys 0.001s
I20260806 22:39:22.584652 9764 ts_tablet_manager.cc:1428] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998: Time spent starting tablet: real 0.017s user 0.008s sys 0.008s
I20260806 22:39:22.585826 9552 heartbeater.cc:503] Master 127.15.198.190:41277 was elected leader, sending a full tablet report...
I20260806 22:39:22.586539 9767 ts_tablet_manager.cc:1428] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373: Time spent starting tablet: real 0.015s user 0.006s sys 0.008s
I20260806 22:39:22.677757 9772 raft_consensus.cc:482] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:39:22.678232 9772 raft_consensus.cc:504] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f643f074dbad48378878d182cc9f7373" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 41075 } } peers { permanent_uuid: "a8d2695ad18844638924852fe7743ed1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 34617 } } peers { permanent_uuid: "33bc277c32134140b7ec5fd05478a998" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 40365 } }
I20260806 22:39:22.680941 9772 leader_election.cc:288] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f643f074dbad48378878d182cc9f7373 (127.15.198.130:41075), a8d2695ad18844638924852fe7743ed1 (127.15.198.129:34617)
I20260806 22:39:22.687709 9763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:22.681778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:59915 (local address 127.15.198.129:34617)
0806 22:39:22.682000 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:22.682008 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:22.682033 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:39:22.682333 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:22.682338 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:22.682412 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:22.682532 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:22.683054 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.683869 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.685697 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.686572 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.686764 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:22.686850 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:22.687267 (+ 417us) server_negotiation.cc:299] Negotiation successful
0806 22:39:22.687412 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 22:39:22.687726 9744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:22.681531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:34617 (local address 127.15.198.131:59915)
0806 22:39:22.681950 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:22.681976 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:22.682130 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:22.682614 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:22.682619 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:22.682650 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:22.682853 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:22.682862 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.684036 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.684048 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:22.685505 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:22.685519 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.686753 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.686758 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:22.686983 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:22.687092 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 22:39:22.687199 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 22:39:22.687369 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272}
I20260806 22:39:22.688392 9757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:22.681639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:40079 (local address 127.15.198.130:41075)
0806 22:39:22.681923 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:22.681932 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:22.682720 (+ 788us) server_negotiation.cc:407] Connection header received
0806 22:39:22.682963 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:22.682969 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:22.683115 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:22.683226 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:22.683831 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.684556 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.686343 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.687345 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.687513 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:22.687596 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:22.687961 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 22:39:22.688114 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260806 22:39:22.689050 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:22.681572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:41075 (local address 127.15.198.131:40079)
0806 22:39:22.682580 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:22.682599 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:22.682784 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:22.683242 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:22.683246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:22.683272 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:22.683658 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:22.683667 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.684727 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.684731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:22.686136 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:22.686149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.687570 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.687575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:22.687734 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:22.687759 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 22:39:22.688545 (+ 786us) client_negotiation.cc:240] Negotiation successful
0806 22:39:22.688715 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":600,"threads_started":1}
I20260806 22:39:22.689229 9304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b64444b8e06472abf0455fe476ccb81" candidate_uuid: "33bc277c32134140b7ec5fd05478a998" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8d2695ad18844638924852fe7743ed1" is_pre_election: true
I20260806 22:39:22.690030 9304 raft_consensus.cc:2454] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33bc277c32134140b7ec5fd05478a998 in term 0.
W20260806 22:39:22.690411 9742 tablet.cc:2369] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:39:22.690425 9497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b64444b8e06472abf0455fe476ccb81" candidate_uuid: "33bc277c32134140b7ec5fd05478a998" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f643f074dbad48378878d182cc9f7373" is_pre_election: true
I20260806 22:39:22.690964 9589 leader_election.cc:302] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33bc277c32134140b7ec5fd05478a998, a8d2695ad18844638924852fe7743ed1; no voters:
I20260806 22:39:22.691157 9497 raft_consensus.cc:2454] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33bc277c32134140b7ec5fd05478a998 in term 0.
I20260806 22:39:22.691596 9772 raft_consensus.cc:2781] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:39:22.691792 9772 raft_consensus.cc:482] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:39:22.691932 9772 raft_consensus.cc:3037] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:22.694738 9772 raft_consensus.cc:504] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f643f074dbad48378878d182cc9f7373" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 41075 } } peers { permanent_uuid: "a8d2695ad18844638924852fe7743ed1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 34617 } } peers { permanent_uuid: "33bc277c32134140b7ec5fd05478a998" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 40365 } }
I20260806 22:39:22.695528 9772 leader_election.cc:288] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [CANDIDATE]: Term 1 election: Requested vote from peers f643f074dbad48378878d182cc9f7373 (127.15.198.130:41075), a8d2695ad18844638924852fe7743ed1 (127.15.198.129:34617)
I20260806 22:39:22.696189 9304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b64444b8e06472abf0455fe476ccb81" candidate_uuid: "33bc277c32134140b7ec5fd05478a998" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8d2695ad18844638924852fe7743ed1"
I20260806 22:39:22.696249 9497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b64444b8e06472abf0455fe476ccb81" candidate_uuid: "33bc277c32134140b7ec5fd05478a998" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f643f074dbad48378878d182cc9f7373"
I20260806 22:39:22.696451 9304 raft_consensus.cc:3037] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:22.696589 9497 raft_consensus.cc:3037] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:22.699303 9304 raft_consensus.cc:2454] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33bc277c32134140b7ec5fd05478a998 in term 1.
I20260806 22:39:22.699604 9497 raft_consensus.cc:2454] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33bc277c32134140b7ec5fd05478a998 in term 1.
I20260806 22:39:22.699875 9589 leader_election.cc:302] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33bc277c32134140b7ec5fd05478a998, a8d2695ad18844638924852fe7743ed1; no voters:
I20260806 22:39:22.700271 9772 raft_consensus.cc:2781] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:22.700875 9772 raft_consensus.cc:686] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [term 1 LEADER]: Becoming Leader. State: Replica: 33bc277c32134140b7ec5fd05478a998, State: Running, Role: LEADER
I20260806 22:39:22.701496 9772 consensus_queue.cc:237] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f643f074dbad48378878d182cc9f7373" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 41075 } } peers { permanent_uuid: "a8d2695ad18844638924852fe7743ed1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 34617 } } peers { permanent_uuid: "33bc277c32134140b7ec5fd05478a998" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 40365 } }
I20260806 22:39:22.707559 9116 catalog_manager.cc:5495] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33bc277c32134140b7ec5fd05478a998 (127.15.198.131). New cstate: current_term: 1 leader_uuid: "33bc277c32134140b7ec5fd05478a998" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f643f074dbad48378878d182cc9f7373" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8d2695ad18844638924852fe7743ed1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 34617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33bc277c32134140b7ec5fd05478a998" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 40365 } health_report { overall_health: HEALTHY } } }
I20260806 22:39:22.734987 9762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:22.728003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36816 (local address 127.15.198.131:40365)
0806 22:39:22.728226 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:22.728231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:22.728336 (+ 105us) server_negotiation.cc:407] Connection header received
0806 22:39:22.728575 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:22.728579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:22.728684 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:22.728852 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:39:22.729304 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.730143 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.731994 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.732512 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.732636 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:22.733445 (+ 809us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:39:22.734325 (+ 880us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:39:22.734458 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:22.734622 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 22:39:22.734713 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 22:39:22.738466 9371 tablet.cc:2369] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:39:22.746378 9497 raft_consensus.cc:1261] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373 [term 1 FOLLOWER]: Refusing update from remote peer 33bc277c32134140b7ec5fd05478a998: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:22.746428 9304 raft_consensus.cc:1261] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1 [term 1 FOLLOWER]: Refusing update from remote peer 33bc277c32134140b7ec5fd05478a998: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:22.747777 9776 consensus_queue.cc:1035] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8d2695ad18844638924852fe7743ed1" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 34617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:22.748353 9772 consensus_queue.cc:1035] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f643f074dbad48378878d182cc9f7373" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:22.781662 9115 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:39:22.782099 9115 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:39:22.787604 9115 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 099d0a55d7004612940bd693e18fd1bf: Sending DeleteTablet for 3 replicas of tablet 9b64444b8e06472abf0455fe476ccb81
I20260806 22:39:22.789392 9462 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b64444b8e06472abf0455fe476ccb81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:22 UTC) from {username='slave'} at 127.0.0.1:59370
I20260806 22:39:22.789388 9277 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b64444b8e06472abf0455fe476ccb81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:22 UTC) from {username='slave'} at 127.0.0.1:50546
I20260806 22:39:22.790067 9650 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b64444b8e06472abf0455fe476ccb81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:22 UTC) from {username='slave'} at 127.0.0.1:36814
I20260806 22:39:22.792191 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 9193
I20260806 22:39:22.801661 9795 tablet_replica.cc:330] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1: stopping tablet replica
I20260806 22:39:22.802628 9795 raft_consensus.cc:2229] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:39:22.803622 9795 raft_consensus.cc:2258] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:22.809145 9797 tablet_replica.cc:330] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998: stopping tablet replica
I20260806 22:39:22.809712 9796 tablet_replica.cc:330] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373: stopping tablet replica
I20260806 22:39:22.810420 9796 raft_consensus.cc:2229] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:39:22.811268 9796 raft_consensus.cc:2258] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:22.812145 9795 ts_tablet_manager.cc:1905] T 9b64444b8e06472abf0455fe476ccb81 P a8d2695ad18844638924852fe7743ed1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:22.812713 9797 raft_consensus.cc:2229] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:39:22.813179 9763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:22.793805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50558 (local address 127.15.198.129:34617)
0806 22:39:22.794206 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:22.794216 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:22.800241 (+ 6025us) server_negotiation.cc:407] Connection header received
0806 22:39:22.800521 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:22.800532 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:22.800725 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:22.800880 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:22.802119 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.803618 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.806323 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:22.807166 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:22.807301 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:22.810369 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:22.810387 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:22.810430 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:22.811562 (+ 1132us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:22.811653 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:22.811691 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:22.811826 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:22.812479 (+ 653us) server_negotiation.cc:299] Negotiation successful
0806 22:39:22.812697 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 22:39:22.813597 9797 raft_consensus.cc:2258] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:22.815876 9365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:22.816372 9796 ts_tablet_manager.cc:1905] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:22.820380 9797 ts_tablet_manager.cc:1905] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:22.825809 9796 ts_tablet_manager.cc:1918] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:22.826056 9796 log.cc:1196] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/9b64444b8e06472abf0455fe476ccb81
I20260806 22:39:22.826782 9796 ts_tablet_manager.cc:1939] T 9b64444b8e06472abf0455fe476ccb81 P f643f074dbad48378878d182cc9f7373: Deleting consensus metadata
I20260806 22:39:22.829614 9098 catalog_manager.cc:4841] TS f643f074dbad48378878d182cc9f7373 (127.15.198.130:41075): tablet 9b64444b8e06472abf0455fe476ccb81 (table test-table [id=796c843e4fe14c54abe3dbf57845ff74]) successfully deleted
I20260806 22:39:22.830470 9797 ts_tablet_manager.cc:1918] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:22.830734 9797 log.cc:1196] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/9b64444b8e06472abf0455fe476ccb81
I20260806 22:39:22.831516 9797 ts_tablet_manager.cc:1939] T 9b64444b8e06472abf0455fe476ccb81 P 33bc277c32134140b7ec5fd05478a998: Deleting consensus metadata
I20260806 22:39:22.834159 9097 catalog_manager.cc:4841] TS 33bc277c32134140b7ec5fd05478a998 (127.15.198.131:40365): tablet 9b64444b8e06472abf0455fe476ccb81 (table test-table [id=796c843e4fe14c54abe3dbf57845ff74]) successfully deleted
I20260806 22:39:22.995453 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 9193
I20260806 22:39:23.025039 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 9375
W20260806 22:39:23.025049 9100 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.198.129:34617 (error 108)
I20260806 22:39:23.027575 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:23.026949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:34617 (local address 127.0.0.1:50566)
0806 22:39:23.027204 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:23.027353 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:34617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260806 22:39:23.027869 9100 catalog_manager.cc:4584] TS a8d2695ad18844638924852fe7743ed1 (127.15.198.129:34617): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9b64444b8e06472abf0455fe476ccb81: Network error: Client connection negotiation failed: client connection to 127.15.198.129:34617: connect: Connection refused (error 111)
I20260806 22:39:23.034644 9757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:23.026552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59380 (local address 127.15.198.130:41075)
0806 22:39:23.026785 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:23.026790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:23.026810 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:39:23.026883 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:23.026887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:23.026989 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:23.027189 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:23.027821 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:23.028763 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:23.030789 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:23.031432 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:23.031559 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:23.032949 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:23.032963 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:23.032984 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:23.033759 (+ 775us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:23.033820 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:23.033842 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:23.034031 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:23.034100 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 22:39:23.034218 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 22:39:23.035605 9547 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:23.082037 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:23.081338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:34617 (local address 127.0.0.1:50572)
0806 22:39:23.081661 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:23.081801 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:34617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260806 22:39:23.083626 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:23.083106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:34617 (local address 127.0.0.1:50580)
0806 22:39:23.083300 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:23.083419 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:34617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 22:39:23.140978 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 9375
I20260806 22:39:23.149376 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:23.148605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:34617 (local address 127.0.0.1:50592)
0806 22:39:23.148945 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:23.149107 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:34617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260806 22:39:23.150684 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:23.150273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:34617 (local address 127.0.0.1:50596)
0806 22:39:23.150453 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:23.150535 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:34617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 22:39:23.178864 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 9559
I20260806 22:39:23.189005 9762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:23.179965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36826 (local address 127.15.198.131:40365)
0806 22:39:23.180204 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:23.180209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:23.180349 (+ 140us) server_negotiation.cc:407] Connection header received
0806 22:39:23.180625 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:23.180630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:23.180769 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:23.180869 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:23.181512 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:23.182403 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:23.184757 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:23.185276 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:23.185399 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:23.186962 (+ 1563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:23.186973 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:23.187062 (+ 89us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:23.187893 (+ 831us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:23.188020 (+ 127us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:23.188052 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:23.188176 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:23.188396 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 22:39:23.188542 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 22:39:23.190032 9732 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:23.232051 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:23.231389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:34617 (local address 127.0.0.1:50612)
0806 22:39:23.231669 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:23.231797 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:34617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260806 22:39:23.233417 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:23.232975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:34617 (local address 127.0.0.1:50624)
0806 22:39:23.233201 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:23.233285 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:34617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260806 22:39:23.336624 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 9559
I20260806 22:39:23.373586 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 9077
I20260806 22:39:23.375290 9155 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:23.509274 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:23.508572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:34617 (local address 127.0.0.1:50640)
0806 22:39:23.508809 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:23.509005 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:34617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 22:39:23.509280 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 9077
2026-08-06T22:39:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3767 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:39:23 dist-test-slave-h3ft krb5kdc[9807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:39:23 dist-test-slave-h3ft krb5kdc[9807](info): set up 2 sockets
Aug 06 22:39:23 dist-test-slave-h3ft krb5kdc[9807](info): commencing operation
krb5kdc: starting...
W20260806 22:39:24.942798 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.374s user 0.000s sys 0.065s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:39:25 dist-test-slave-h3ft krb5kdc[9807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:39:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:39:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:39:25.379966 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:36219
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:36219
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:25.840479 9828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:25.840915 9828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:25.841032 9828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:25.849834 9828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:25.850018 9828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:25.850070 9828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:25.850136 9828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:25.850186 9828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:25.862231 9828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:36219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:36219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:25.864120 9828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:25.865844 9828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:25.881422 9838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:25.881358 9837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:25.881492 9840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:25.883139 9828 server_base.cc:1034] running on GCE node
I20260806 22:39:25.884305 9828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:25.886399 9828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:25.887681 9828 hybrid_clock.cc:648] HybridClock initialized: now 1786055965887615 us; error 57 us; skew 500 ppm
I20260806 22:39:25.888167 9828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:25 dist-test-slave-h3ft krb5kdc[9807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055965, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:25.892313 9828 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:39:25.894387 9828 webserver.cc:460] Webserver started at http://127.15.198.190:34357/ using document root <none> and password file <none>
I20260806 22:39:25.895457 9828 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:25.895625 9828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:25.896023 9828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:25.899996 9828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "2d3c49c158cc4600a2819b381f64fdf5"
format_stamp: "Formatted at 2026-08-06 22:39:25 on dist-test-slave-h3ft"
server_key: "5677cf09cd66ae4be489dc51e74bf231"
server_key_iv: "d1a524ac41b191b0d2d21f87f816562d"
server_key_version: "encryptionkey@0"
I20260806 22:39:25.901000 9828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "2d3c49c158cc4600a2819b381f64fdf5"
format_stamp: "Formatted at 2026-08-06 22:39:25 on dist-test-slave-h3ft"
server_key: "5677cf09cd66ae4be489dc51e74bf231"
server_key_iv: "d1a524ac41b191b0d2d21f87f816562d"
server_key_version: "encryptionkey@0"
I20260806 22:39:25.908043 9828 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 22:39:25.912644 9849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:25.914438 9828 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:39:25.914677 9828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "2d3c49c158cc4600a2819b381f64fdf5"
format_stamp: "Formatted at 2026-08-06 22:39:25 on dist-test-slave-h3ft"
server_key: "5677cf09cd66ae4be489dc51e74bf231"
server_key_iv: "d1a524ac41b191b0d2d21f87f816562d"
server_key_version: "encryptionkey@0"
I20260806 22:39:25.914938 9828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:25.937824 9828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:25.943710 9828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:25.944159 9828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:25.968889 9828 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:36219
I20260806 22:39:25.968894 9919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:36219 every 8 connection(s)
I20260806 22:39:25.970886 9828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:39:25.976444 9920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:25.977018 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 9828
I20260806 22:39:25.977257 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:39:25.977861 16154 external_mini_cluster.cc:1428] Setting key 7c5de523e74c8461cea3f67bcd61d81b
I20260806 22:39:25.994084 9920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5: Bootstrap starting.
Aug 06 22:39:25 dist-test-slave-h3ft krb5kdc[9807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:39:25.998992 9920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:26.000584 9920 log.cc:824] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:26.004724 9920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5: No bootstrap required, opened a new log
I20260806 22:39:26.008122 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:25.980601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42522 (local address 127.15.198.190:36219)
0806 22:39:25.981666 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:25.981681 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:25.981738 (+ 57us) server_negotiation.cc:407] Connection header received
0806 22:39:25.984245 (+ 2507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:25.984284 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:25.984864 (+ 580us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:25.985318 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:25.987757 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:25.989271 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:25.991386 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:25.992217 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:25.992392 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:25.998404 (+ 6012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:25.998414 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:25.998436 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:25.998499 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:26.002472 (+ 3973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:26.003588 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:26.003597 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:26.003605 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:26.003842 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:26.004356 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:26.004360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:26.004362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:26.005136 (+ 774us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:26.005553 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:26.006045 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 22:39:26.006624 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":785,"thread_start_us":488,"threads_started":1}
I20260806 22:39:26.016364 9920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3c49c158cc4600a2819b381f64fdf5" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36219 } }
I20260806 22:39:26.016996 9920 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:26.017143 9920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d3c49c158cc4600a2819b381f64fdf5, State: Initialized, Role: FOLLOWER
I20260806 22:39:26.018028 9920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3c49c158cc4600a2819b381f64fdf5" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36219 } }
I20260806 22:39:26.018421 9920 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:39:26.018599 9920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:39:26.018755 9920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:26.021620 9920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3c49c158cc4600a2819b381f64fdf5" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36219 } }
I20260806 22:39:26.022357 9920 leader_election.cc:302] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d3c49c158cc4600a2819b381f64fdf5; no voters:
I20260806 22:39:26.023190 9920 leader_election.cc:288] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:39:26.023392 9926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:26.024281 9926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [term 1 LEADER]: Becoming Leader. State: Replica: 2d3c49c158cc4600a2819b381f64fdf5, State: Running, Role: LEADER
I20260806 22:39:26.024978 9926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3c49c158cc4600a2819b381f64fdf5" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36219 } }
I20260806 22:39:26.026328 9920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:26.029603 9929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d3c49c158cc4600a2819b381f64fdf5. Latest consensus state: current_term: 1 leader_uuid: "2d3c49c158cc4600a2819b381f64fdf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3c49c158cc4600a2819b381f64fdf5" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36219 } } }
I20260806 22:39:26.029277 9927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d3c49c158cc4600a2819b381f64fdf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3c49c158cc4600a2819b381f64fdf5" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 36219 } } }
I20260806 22:39:26.030035 9929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:26.030045 9927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:26.036139 9935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:39:26.041739 9935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:39:26.053393 9935 catalog_manager.cc:1347] Generated new cluster ID: 847c4820e5a046ec8ffbeff5f269b878
I20260806 22:39:26.053573 9935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:39:26.078994 9935 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:39:26.081686 9935 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:39:26.098623 9935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5: Generated new TSK 0
I20260806 22:39:26.100069 9935 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
I20260806 22:39:26.204018 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36219
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:26.719254 9954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:26.719669 9954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:26.719913 9954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:26.728317 9954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:26.728472 9954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:26.728690 9954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:39:26.740412 9954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36219
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:26.742704 9954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:26.744908 9954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:26.762306 9965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:26.762210 9962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:26.762298 9967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:26.763414 9954 server_base.cc:1034] running on GCE node
I20260806 22:39:26.764272 9954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:26.765518 9954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:26.766781 9954 hybrid_clock.cc:648] HybridClock initialized: now 1786055966766724 us; error 51 us; skew 500 ppm
I20260806 22:39:26.767239 9954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:26 dist-test-slave-h3ft krb5kdc[9807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055966, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:26.771252 9954 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:39:26.773128 9954 webserver.cc:460] Webserver started at http://127.15.198.129:35337/ using document root <none> and password file <none>
I20260806 22:39:26.774036 9954 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:26.774183 9954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:26.774606 9954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:26.778400 9954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "a9333fc00894494599ec3b2bcbcd6599"
format_stamp: "Formatted at 2026-08-06 22:39:26 on dist-test-slave-h3ft"
server_key: "67499cdbd947b6ad480329a05c42f614"
server_key_iv: "9ece2467781b07c327ee07155c6f66b0"
server_key_version: "encryptionkey@0"
I20260806 22:39:26.779444 9954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "a9333fc00894494599ec3b2bcbcd6599"
format_stamp: "Formatted at 2026-08-06 22:39:26 on dist-test-slave-h3ft"
server_key: "67499cdbd947b6ad480329a05c42f614"
server_key_iv: "9ece2467781b07c327ee07155c6f66b0"
server_key_version: "encryptionkey@0"
I20260806 22:39:26.786485 9954 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260806 22:39:26.791481 9975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:26.793341 9954 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 22:39:26.793598 9954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "a9333fc00894494599ec3b2bcbcd6599"
format_stamp: "Formatted at 2026-08-06 22:39:26 on dist-test-slave-h3ft"
server_key: "67499cdbd947b6ad480329a05c42f614"
server_key_iv: "9ece2467781b07c327ee07155c6f66b0"
server_key_version: "encryptionkey@0"
I20260806 22:39:26.793876 9954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:26.813568 9954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:26.819191 9954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:26.819568 9954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:26.820931 9954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:26.822718 9954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:26.822829 9954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:26.823043 9954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:26.823133 9954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260806 22:39:26.867411 9954 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:45269
I20260806 22:39:26.867463 10126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:45269 every 8 connection(s)
I20260806 22:39:26.869176 9954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:39:26.875555 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 9954
I20260806 22:39:26.875787 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:39:26.876364 16154 external_mini_cluster.cc:1428] Setting key 4d63b6f1f36d9c876229038a7668dc3e
Aug 06 22:39:26 dist-test-slave-h3ft krb5kdc[9807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055966, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:39:26.893416 10129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:26.872519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:43437 (local address 127.15.198.190:36219)
0806 22:39:26.873174 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:26.873180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:26.874326 (+ 1146us) server_negotiation.cc:407] Connection header received
0806 22:39:26.876679 (+ 2353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:26.876686 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:26.876799 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:26.876942 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:26.879036 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:26.879872 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:26.881588 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:26.882039 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:26.882189 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:26.886427 (+ 4238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:26.886433 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:26.886436 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:26.886477 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:26.889510 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:26.890429 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:26.890436 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:26.890439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:26.890595 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:26.891116 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:26.891121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:26.891131 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:26.891476 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:26.891681 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:26.892618 (+ 937us) server_negotiation.cc:299] Negotiation successful
0806 22:39:26.892919 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":319,"threads_started":1}
I20260806 22:39:26.894748 10131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:26.873255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:36219 (local address 127.15.198.129:43437)
0806 22:39:26.874114 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:26.874164 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:26.876316 (+ 2152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:26.877274 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:26.877287 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:26.878009 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:26.878821 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:26.878848 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:26.880048 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:26.880053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:26.881420 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:26.881433 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:26.882310 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:26.882319 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:26.882595 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:26.883481 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:26.883516 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:26.886131 (+ 2615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:26.889697 (+ 3566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:26.889709 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:26.889738 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:26.890283 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:26.890724 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:26.890729 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:26.890733 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:26.890968 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:26.891632 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:26.891644 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:26.892138 (+ 494us) client_negotiation.cc:769] Sending connection context
0806 22:39:26.892472 (+ 334us) client_negotiation.cc:240] Negotiation successful
0806 22:39:26.893213 (+ 741us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":324,"threads_started":1}
I20260806 22:39:26.896896 10127 heartbeater.cc:347] Connected to a master server at 127.15.198.190:36219
I20260806 22:39:26.897365 10127 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:26.898494 10127 heartbeater.cc:511] Master 127.15.198.190:36219 requested a full tablet report, sending...
I20260806 22:39:26.901706 9873 ts_manager.cc:194] Registered new tserver with Master: a9333fc00894494599ec3b2bcbcd6599 (127.15.198.129:45269)
I20260806 22:39:26.904232 9873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:43437
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
I20260806 22:39:26.969061 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36219
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:27.409085 10135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:27.409540 10135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:27.409740 10135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:27.418264 10135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:27.418437 10135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:27.418648 10135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:39:27.430469 10135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36219
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:27.432471 10135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:27.434414 10135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:27.450798 10142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:27.450919 10143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:27.451357 10145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:27.452406 10135 server_base.cc:1034] running on GCE node
I20260806 22:39:27.453334 10135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:27.454766 10135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:27.456025 10135 hybrid_clock.cc:648] HybridClock initialized: now 1786055967455964 us; error 37 us; skew 500 ppm
I20260806 22:39:27.456519 10135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:27 dist-test-slave-h3ft krb5kdc[9807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055967, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:27.460940 10135 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:39:27.463402 10135 webserver.cc:460] Webserver started at http://127.15.198.130:45247/ using document root <none> and password file <none>
I20260806 22:39:27.464708 10135 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:27.464862 10135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:27.465348 10135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:27.469676 10135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "6b475a67fbe34fe6abaa71810948cc45"
format_stamp: "Formatted at 2026-08-06 22:39:27 on dist-test-slave-h3ft"
server_key: "5ea3fbed2e9f7239fdc39cc42a7ced04"
server_key_iv: "417466d39c27d99f1867c06291cdda7a"
server_key_version: "encryptionkey@0"
I20260806 22:39:27.470749 10135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "6b475a67fbe34fe6abaa71810948cc45"
format_stamp: "Formatted at 2026-08-06 22:39:27 on dist-test-slave-h3ft"
server_key: "5ea3fbed2e9f7239fdc39cc42a7ced04"
server_key_iv: "417466d39c27d99f1867c06291cdda7a"
server_key_version: "encryptionkey@0"
I20260806 22:39:27.478041 10135 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 22:39:27.483120 10154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:27.485107 10135 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:39:27.485365 10135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "6b475a67fbe34fe6abaa71810948cc45"
format_stamp: "Formatted at 2026-08-06 22:39:27 on dist-test-slave-h3ft"
server_key: "5ea3fbed2e9f7239fdc39cc42a7ced04"
server_key_iv: "417466d39c27d99f1867c06291cdda7a"
server_key_version: "encryptionkey@0"
I20260806 22:39:27.485656 10135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:27.504077 10135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:27.510033 10135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:27.510422 10135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:27.511750 10135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:27.514743 10135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:27.514895 10135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:27.515093 10135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:27.515192 10135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 22:39:27.557185 10135 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:45555
I20260806 22:39:27.557337 10312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:45555 every 8 connection(s)
I20260806 22:39:27.558965 10135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:39:27.567365 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 10135
I20260806 22:39:27.567641 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:39:27.568219 16154 external_mini_cluster.cc:1428] Setting key 7489d1c704b55813d7e9b6ee0056c72e
Aug 06 22:39:27 dist-test-slave-h3ft krb5kdc[9807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055967, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:39:27.586948 10315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:27.562462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:58621 (local address 127.15.198.190:36219)
0806 22:39:27.563059 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:27.563067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:27.564494 (+ 1427us) server_negotiation.cc:407] Connection header received
0806 22:39:27.566656 (+ 2162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:27.566662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:27.566752 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:27.566943 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:27.569443 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:27.570325 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:27.572314 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:27.572821 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:27.572988 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:27.577737 (+ 4749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:27.577747 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:27.577755 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:27.577814 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:27.582314 (+ 4500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:27.583560 (+ 1246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:27.583570 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:27.583576 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:27.583779 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:27.584403 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:27.584412 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:27.584417 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:27.584901 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:27.585079 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:27.586047 (+ 968us) server_negotiation.cc:299] Negotiation successful
0806 22:39:27.586384 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":258,"threads_started":1}
I20260806 22:39:27.588172 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:27.563272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:36219 (local address 127.15.198.130:58621)
0806 22:39:27.564252 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:27.564315 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:27.566333 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:27.567300 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:27.567311 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:27.568182 (+ 871us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:27.569174 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:27.569212 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:27.570544 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:27.570548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:27.572120 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:27.572135 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:27.573125 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:27.573134 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:27.573438 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:27.574325 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:27.574356 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:27.577370 (+ 3014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:27.582571 (+ 5201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:27.582582 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:27.582604 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:27.583383 (+ 779us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:27.583973 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:27.583979 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:27.583981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:27.584260 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:27.585067 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:27.585076 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:27.585569 (+ 493us) client_negotiation.cc:769] Sending connection context
0806 22:39:27.585861 (+ 292us) client_negotiation.cc:240] Negotiation successful
0806 22:39:27.586465 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":366,"threads_started":1}
I20260806 22:39:27.590091 10313 heartbeater.cc:347] Connected to a master server at 127.15.198.190:36219
I20260806 22:39:27.590564 10313 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:27.591678 10313 heartbeater.cc:511] Master 127.15.198.190:36219 requested a full tablet report, sending...
I20260806 22:39:27.593523 9873 ts_manager.cc:194] Registered new tserver with Master: 6b475a67fbe34fe6abaa71810948cc45 (127.15.198.130:45555)
I20260806 22:39:27.594494 9873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:58621
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
I20260806 22:39:27.663167 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36219
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:39:27.908483 10127 heartbeater.cc:503] Master 127.15.198.190:36219 was elected leader, sending a full tablet report...
W20260806 22:39:28.149278 10321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:28.149793 10321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:28.149991 10321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:28.158470 10321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:28.158715 10321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:28.158952 10321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:39:28.170943 10321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:36219
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:28.173089 10321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:28.174880 10321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:28.192376 10328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:28.192297 10329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:28.192334 10331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:28.194044 10321 server_base.cc:1034] running on GCE node
I20260806 22:39:28.194908 10321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:28.196308 10321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:28.197582 10321 hybrid_clock.cc:648] HybridClock initialized: now 1786055968197515 us; error 51 us; skew 500 ppm
I20260806 22:39:28.198055 10321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:28 dist-test-slave-h3ft krb5kdc[9807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055968, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:28.202561 10321 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:39:28.204716 10321 webserver.cc:460] Webserver started at http://127.15.198.131:38107/ using document root <none> and password file <none>
I20260806 22:39:28.205706 10321 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:28.205899 10321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:28.206331 10321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:28.210353 10321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "a866f84adbd54df6b7e608c570411981"
format_stamp: "Formatted at 2026-08-06 22:39:28 on dist-test-slave-h3ft"
server_key: "3dc61e825e64ae97a3a331283fe7e027"
server_key_iv: "1fe878bff8de49b6a8e683858cbb0d83"
server_key_version: "encryptionkey@0"
I20260806 22:39:28.211617 10321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "a866f84adbd54df6b7e608c570411981"
format_stamp: "Formatted at 2026-08-06 22:39:28 on dist-test-slave-h3ft"
server_key: "3dc61e825e64ae97a3a331283fe7e027"
server_key_iv: "1fe878bff8de49b6a8e683858cbb0d83"
server_key_version: "encryptionkey@0"
I20260806 22:39:28.219040 10321 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260806 22:39:28.223769 10344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:28.225620 10321 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:39:28.225920 10321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "a866f84adbd54df6b7e608c570411981"
format_stamp: "Formatted at 2026-08-06 22:39:28 on dist-test-slave-h3ft"
server_key: "3dc61e825e64ae97a3a331283fe7e027"
server_key_iv: "1fe878bff8de49b6a8e683858cbb0d83"
server_key_version: "encryptionkey@0"
I20260806 22:39:28.226199 10321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:28.243429 10321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:28.248840 10321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:28.249223 10321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:28.250502 10321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:28.252276 10321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:28.252393 10321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:28.252579 10321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:28.252665 10321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:28.292537 10321 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:46387
I20260806 22:39:28.292595 10502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:46387 every 8 connection(s)
I20260806 22:39:28.294271 10321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:39:28.295758 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 10321
I20260806 22:39:28.296013 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:39:28.296732 16154 external_mini_cluster.cc:1428] Setting key 17ec34a8744e84bd89891b0215cdca0d
Aug 06 22:39:28 dist-test-slave-h3ft krb5kdc[9807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055968, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:39:28.319799 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:28.297818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:41395 (local address 127.15.198.190:36219)
0806 22:39:28.298478 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:28.298489 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:28.299873 (+ 1384us) server_negotiation.cc:407] Connection header received
0806 22:39:28.302113 (+ 2240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:28.302117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:28.302199 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:28.302305 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:28.304314 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.305230 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.306918 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.307489 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.307622 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:28.312072 (+ 4450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:28.312079 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:28.312082 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:28.312122 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:28.315874 (+ 3752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:28.316837 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:28.316842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:28.316845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:28.317020 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:28.317664 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:28.317667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:28.317669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:28.318036 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:28.318161 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:28.319120 (+ 959us) server_negotiation.cc:299] Negotiation successful
0806 22:39:28.319428 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":332,"threads_started":1}
I20260806 22:39:28.321275 10507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:28.298579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:36219 (local address 127.15.198.131:41395)
0806 22:39:28.299664 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:28.299721 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:28.301804 (+ 2083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:28.302581 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:28.302592 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:28.303355 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:28.304099 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:28.304122 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.305379 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.305384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:28.306768 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:28.306782 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.307805 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.307811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:28.308083 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:28.308968 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:28.308995 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:28.311792 (+ 2797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:28.316064 (+ 4272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:28.316073 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:28.316094 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:28.316691 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:28.317148 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:28.317152 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:28.317155 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:28.317546 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:28.318157 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:28.318165 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:28.318608 (+ 443us) client_negotiation.cc:769] Sending connection context
0806 22:39:28.318884 (+ 276us) client_negotiation.cc:240] Negotiation successful
0806 22:39:28.319637 (+ 753us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":413,"threads_started":1}
I20260806 22:39:28.323349 10504 heartbeater.cc:347] Connected to a master server at 127.15.198.190:36219
I20260806 22:39:28.323863 10504 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:28.324972 10504 heartbeater.cc:511] Master 127.15.198.190:36219 requested a full tablet report, sending...
I20260806 22:39:28.327196 9872 ts_manager.cc:194] Registered new tserver with Master: a866f84adbd54df6b7e608c570411981 (127.15.198.131:46387)
I20260806 22:39:28.328431 9872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:41395
I20260806 22:39:28.340142 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:39:28.360417 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:28.345093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42528 (local address 127.15.198.190:36219)
0806 22:39:28.345308 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:28.345313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:28.345912 (+ 599us) server_negotiation.cc:407] Connection header received
0806 22:39:28.346110 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:28.346115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:28.346216 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:28.346318 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:28.348078 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.348890 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.350284 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.350707 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.350806 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:28.353874 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:28.353880 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:28.353883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:28.353924 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:28.356902 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:28.357841 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:28.357845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:28.357848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:28.358065 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:28.358562 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:28.358567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:28.358572 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:28.359117 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:28.359436 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:28.359915 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 22:39:28.360126 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 22:39:28.366043 9872 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:39:28.374538 9872 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:39:28.418488 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:28.392387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34468 (local address 127.15.198.131:46387)
0806 22:39:28.406898 (+ 14511us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:28.406918 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:28.407575 (+ 657us) server_negotiation.cc:407] Connection header received
0806 22:39:28.407724 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:28.407733 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:28.408133 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:28.408386 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:28.409268 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.411401 (+ 2133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.414220 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.416346 (+ 2126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.416607 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:28.416879 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:28.417752 (+ 873us) server_negotiation.cc:299] Negotiation successful
0806 22:39:28.418036 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14185,"thread_start_us":13879,"threads_started":1}
I20260806 22:39:28.418532 10519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:28.392099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:46387 (local address 127.0.0.1:34468)
0806 22:39:28.407239 (+ 15140us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:28.407263 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:28.407422 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:28.408467 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:28.408479 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:28.408517 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:28.409039 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:28.409054 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.411667 (+ 2613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.411672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:28.413991 (+ 2319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:28.414008 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.417085 (+ 3077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.417094 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:28.417390 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:28.417469 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 22:39:28.417675 (+ 206us) client_negotiation.cc:240] Negotiation successful
0806 22:39:28.417967 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14985,"thread_start_us":12883,"threads_started":1}
I20260806 22:39:28.419935 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:28.407770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:45269 (local address 127.0.0.1:43030)
0806 22:39:28.410687 (+ 2917us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:28.410724 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:28.410939 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:28.412306 (+ 1367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:28.412312 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:28.412353 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:28.412960 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:28.412978 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.414776 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.414784 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:28.416739 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:28.416753 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.418876 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.418881 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:28.419155 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:28.419193 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:39:28.419318 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 22:39:28.419518 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":407,"threads_started":1}
I20260806 22:39:28.420580 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:28.391880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32960 (local address 127.15.198.130:45555)
0806 22:39:28.408873 (+ 16993us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:28.408885 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:28.408922 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:39:28.409077 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:28.409087 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:28.409340 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:28.409588 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:28.411359 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.413356 (+ 1997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.416546 (+ 3190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.418512 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.418722 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:28.418965 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:28.419860 (+ 895us) server_negotiation.cc:299] Negotiation successful
0806 22:39:28.420108 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15545,"thread_start_us":15324,"threads_started":1}
I20260806 22:39:28.420796 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:28.391526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:45555 (local address 127.0.0.1:32960)
0806 22:39:28.404270 (+ 12744us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:28.404356 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:28.404640 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:28.410354 (+ 5714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:28.410360 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:28.410400 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:28.411054 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:28.411071 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.413529 (+ 2458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.413535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:28.416247 (+ 2712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:28.416274 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.419282 (+ 3008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.419290 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:28.419563 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:28.419607 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:39:28.419713 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 22:39:28.419915 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12311,"thread_start_us":15345,"threads_started":1}
I20260806 22:39:28.421101 10525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:28.407903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43030 (local address 127.15.198.129:45269)
0806 22:39:28.410398 (+ 2495us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:28.410409 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:28.411110 (+ 701us) server_negotiation.cc:407] Connection header received
0806 22:39:28.411361 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:28.411370 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:28.411666 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:28.411944 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:28.413186 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.414622 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.416967 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.418506 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.420089 (+ 1583us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:28.420315 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:28.420410 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:39:28.420640 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2302,"thread_start_us":2063,"threads_started":1}
I20260806 22:39:28.424612 10409 tablet_service.cc:1467] Processing CreateTablet for tablet e9a9beace0ae49eb81b32609e5654bb9 (DEFAULT_TABLE table=test-table [id=dad3e89f73414adfb3c6a71f2b0a103d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:28.425426 10221 tablet_service.cc:1467] Processing CreateTablet for tablet e9a9beace0ae49eb81b32609e5654bb9 (DEFAULT_TABLE table=test-table [id=dad3e89f73414adfb3c6a71f2b0a103d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:28.425426 10040 tablet_service.cc:1467] Processing CreateTablet for tablet e9a9beace0ae49eb81b32609e5654bb9 (DEFAULT_TABLE table=test-table [id=dad3e89f73414adfb3c6a71f2b0a103d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:28.426586 10409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9a9beace0ae49eb81b32609e5654bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:28.427268 10221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9a9beace0ae49eb81b32609e5654bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:28.427322 10040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9a9beace0ae49eb81b32609e5654bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:28.441826 10527 tablet_bootstrap.cc:492] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981: Bootstrap starting.
I20260806 22:39:28.446779 10527 tablet_bootstrap.cc:654] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:28.448154 10527 log.cc:824] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:28.451164 10532 tablet_bootstrap.cc:492] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599: Bootstrap starting.
I20260806 22:39:28.451591 10529 tablet_bootstrap.cc:492] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45: Bootstrap starting.
I20260806 22:39:28.452755 10527 tablet_bootstrap.cc:492] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981: No bootstrap required, opened a new log
I20260806 22:39:28.453095 10527 ts_tablet_manager.cc:1397] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.002s
I20260806 22:39:28.456530 10532 tablet_bootstrap.cc:654] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:28.457480 10529 tablet_bootstrap.cc:654] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:28.458460 10532 log.cc:824] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:28.459403 10529 log.cc:824] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:28.463286 10527 raft_consensus.cc:348] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b475a67fbe34fe6abaa71810948cc45" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45555 } } peers { permanent_uuid: "a866f84adbd54df6b7e608c570411981" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 46387 } } peers { permanent_uuid: "a9333fc00894494599ec3b2bcbcd6599" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45269 } }
I20260806 22:39:28.463908 10527 raft_consensus.cc:374] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:28.464076 10527 raft_consensus.cc:729] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a866f84adbd54df6b7e608c570411981, State: Initialized, Role: FOLLOWER
I20260806 22:39:28.464161 10532 tablet_bootstrap.cc:492] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599: No bootstrap required, opened a new log
I20260806 22:39:28.464159 10529 tablet_bootstrap.cc:492] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45: No bootstrap required, opened a new log
I20260806 22:39:28.464551 10532 ts_tablet_manager.cc:1397] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.001s
I20260806 22:39:28.464655 10529 ts_tablet_manager.cc:1397] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260806 22:39:28.464928 10527 consensus_queue.cc:260] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b475a67fbe34fe6abaa71810948cc45" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45555 } } peers { permanent_uuid: "a866f84adbd54df6b7e608c570411981" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 46387 } } peers { permanent_uuid: "a9333fc00894494599ec3b2bcbcd6599" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45269 } }
I20260806 22:39:28.466478 10504 heartbeater.cc:503] Master 127.15.198.190:36219 was elected leader, sending a full tablet report...
I20260806 22:39:28.467267 10527 ts_tablet_manager.cc:1428] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981: Time spent starting tablet: real 0.014s user 0.012s sys 0.003s
I20260806 22:39:28.476336 10532 raft_consensus.cc:348] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b475a67fbe34fe6abaa71810948cc45" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45555 } } peers { permanent_uuid: "a866f84adbd54df6b7e608c570411981" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 46387 } } peers { permanent_uuid: "a9333fc00894494599ec3b2bcbcd6599" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45269 } }
I20260806 22:39:28.476334 10529 raft_consensus.cc:348] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b475a67fbe34fe6abaa71810948cc45" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45555 } } peers { permanent_uuid: "a866f84adbd54df6b7e608c570411981" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 46387 } } peers { permanent_uuid: "a9333fc00894494599ec3b2bcbcd6599" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45269 } }
I20260806 22:39:28.477023 10529 raft_consensus.cc:374] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:28.477072 10532 raft_consensus.cc:374] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:28.477205 10532 raft_consensus.cc:729] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9333fc00894494599ec3b2bcbcd6599, State: Initialized, Role: FOLLOWER
I20260806 22:39:28.477205 10529 raft_consensus.cc:729] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b475a67fbe34fe6abaa71810948cc45, State: Initialized, Role: FOLLOWER
I20260806 22:39:28.478014 10532 consensus_queue.cc:260] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b475a67fbe34fe6abaa71810948cc45" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45555 } } peers { permanent_uuid: "a866f84adbd54df6b7e608c570411981" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 46387 } } peers { permanent_uuid: "a9333fc00894494599ec3b2bcbcd6599" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45269 } }
I20260806 22:39:28.478021 10529 consensus_queue.cc:260] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b475a67fbe34fe6abaa71810948cc45" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45555 } } peers { permanent_uuid: "a866f84adbd54df6b7e608c570411981" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 46387 } } peers { permanent_uuid: "a9333fc00894494599ec3b2bcbcd6599" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45269 } }
I20260806 22:39:28.479874 10313 heartbeater.cc:503] Master 127.15.198.190:36219 was elected leader, sending a full tablet report...
I20260806 22:39:28.480572 10532 ts_tablet_manager.cc:1428] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599: Time spent starting tablet: real 0.016s user 0.009s sys 0.005s
I20260806 22:39:28.480710 10529 ts_tablet_manager.cc:1428] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45: Time spent starting tablet: real 0.016s user 0.004s sys 0.011s
W20260806 22:39:28.546548 10505 tablet.cc:2369] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:39:28.561866 10314 tablet.cc:2369] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:39:28.572542 10537 raft_consensus.cc:482] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:39:28.572947 10537 raft_consensus.cc:504] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b475a67fbe34fe6abaa71810948cc45" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45555 } } peers { permanent_uuid: "a866f84adbd54df6b7e608c570411981" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 46387 } } peers { permanent_uuid: "a9333fc00894494599ec3b2bcbcd6599" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45269 } }
I20260806 22:39:28.575248 10537 leader_election.cc:288] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b475a67fbe34fe6abaa71810948cc45 (127.15.198.130:45555), a866f84adbd54df6b7e608c570411981 (127.15.198.131:46387)
I20260806 22:39:28.583806 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:28.575996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:50189 (local address 127.15.198.131:46387)
0806 22:39:28.576273 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:28.576281 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:28.577214 (+ 933us) server_negotiation.cc:407] Connection header received
0806 22:39:28.577455 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:28.577459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:28.577534 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:28.577650 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:28.578393 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.579356 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.581643 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.582539 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.582773 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:28.582884 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:28.583286 (+ 402us) server_negotiation.cc:299] Negotiation successful
0806 22:39:28.583462 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 22:39:28.583806 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:28.575947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:35729 (local address 127.15.198.130:45555)
0806 22:39:28.576245 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:28.576254 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:28.577332 (+ 1078us) server_negotiation.cc:407] Connection header received
0806 22:39:28.577557 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:28.577566 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:28.577637 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:28.577746 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:28.578466 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.579371 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.581618 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.582546 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.582773 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:28.582875 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:28.583287 (+ 412us) server_negotiation.cc:299] Negotiation successful
0806 22:39:28.583465 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260806 22:39:28.584041 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:28.575940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:46387 (local address 127.15.198.129:50189)
0806 22:39:28.577055 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:28.577075 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:28.577261 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:28.577677 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:28.577682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:28.577733 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:28.578207 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:28.578222 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.579515 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.579522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:28.581421 (+ 1899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:28.581453 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.582879 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.582886 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:28.583108 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:28.583134 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 22:39:28.583244 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 22:39:28.583483 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"mutex_wait_us":91,"thread_start_us":630,"threads_started":1}
I20260806 22:39:28.584067 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:28.575999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:45555 (local address 127.15.198.129:35729)
0806 22:39:28.577191 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:28.577211 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:28.577398 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:28.577760 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:28.577764 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:28.577787 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:28.578274 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:28.578285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.579518 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.579524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:28.581421 (+ 1897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:28.581438 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.582824 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.582831 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:28.583085 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:28.583119 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 22:39:28.583238 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:39:28.583483 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1059,"mutex_wait_us":178,"thread_start_us":684,"threads_started":1}
I20260806 22:39:28.585788 10435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9a9beace0ae49eb81b32609e5654bb9" candidate_uuid: "a9333fc00894494599ec3b2bcbcd6599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a866f84adbd54df6b7e608c570411981" is_pre_election: true
I20260806 22:39:28.585781 10248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9a9beace0ae49eb81b32609e5654bb9" candidate_uuid: "a9333fc00894494599ec3b2bcbcd6599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b475a67fbe34fe6abaa71810948cc45" is_pre_election: true
I20260806 22:39:28.586625 10435 raft_consensus.cc:2454] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9333fc00894494599ec3b2bcbcd6599 in term 0.
I20260806 22:39:28.586628 10248 raft_consensus.cc:2454] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9333fc00894494599ec3b2bcbcd6599 in term 0.
I20260806 22:39:28.587636 9977 leader_election.cc:302] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b475a67fbe34fe6abaa71810948cc45, a9333fc00894494599ec3b2bcbcd6599; no voters:
I20260806 22:39:28.588124 10537 raft_consensus.cc:2781] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:39:28.588315 10537 raft_consensus.cc:482] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:39:28.588455 10537 raft_consensus.cc:3037] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:28.591599 10537 raft_consensus.cc:504] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b475a67fbe34fe6abaa71810948cc45" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45555 } } peers { permanent_uuid: "a866f84adbd54df6b7e608c570411981" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 46387 } } peers { permanent_uuid: "a9333fc00894494599ec3b2bcbcd6599" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45269 } }
I20260806 22:39:28.592429 10537 leader_election.cc:288] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [CANDIDATE]: Term 1 election: Requested vote from peers 6b475a67fbe34fe6abaa71810948cc45 (127.15.198.130:45555), a866f84adbd54df6b7e608c570411981 (127.15.198.131:46387)
I20260806 22:39:28.593068 10248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9a9beace0ae49eb81b32609e5654bb9" candidate_uuid: "a9333fc00894494599ec3b2bcbcd6599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b475a67fbe34fe6abaa71810948cc45"
I20260806 22:39:28.593091 10435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9a9beace0ae49eb81b32609e5654bb9" candidate_uuid: "a9333fc00894494599ec3b2bcbcd6599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a866f84adbd54df6b7e608c570411981"
I20260806 22:39:28.593354 10435 raft_consensus.cc:3037] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:28.593348 10248 raft_consensus.cc:3037] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:28.596048 10435 raft_consensus.cc:2454] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9333fc00894494599ec3b2bcbcd6599 in term 1.
I20260806 22:39:28.596071 10248 raft_consensus.cc:2454] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9333fc00894494599ec3b2bcbcd6599 in term 1.
I20260806 22:39:28.596602 9979 leader_election.cc:302] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6b475a67fbe34fe6abaa71810948cc45, a866f84adbd54df6b7e608c570411981, a9333fc00894494599ec3b2bcbcd6599; no voters:
I20260806 22:39:28.596993 10537 raft_consensus.cc:2781] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:28.597520 10537 raft_consensus.cc:686] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [term 1 LEADER]: Becoming Leader. State: Replica: a9333fc00894494599ec3b2bcbcd6599, State: Running, Role: LEADER
I20260806 22:39:28.598237 10537 consensus_queue.cc:237] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b475a67fbe34fe6abaa71810948cc45" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45555 } } peers { permanent_uuid: "a866f84adbd54df6b7e608c570411981" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 46387 } } peers { permanent_uuid: "a9333fc00894494599ec3b2bcbcd6599" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45269 } }
I20260806 22:39:28.604107 9873 catalog_manager.cc:5495] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9333fc00894494599ec3b2bcbcd6599 (127.15.198.129). New cstate: current_term: 1 leader_uuid: "a9333fc00894494599ec3b2bcbcd6599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b475a67fbe34fe6abaa71810948cc45" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a866f84adbd54df6b7e608c570411981" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 46387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9333fc00894494599ec3b2bcbcd6599" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 45269 } health_report { overall_health: HEALTHY } } }
W20260806 22:39:28.622742 10128 tablet.cc:2369] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:39:28.653215 10525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:28.646505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45006 (local address 127.15.198.129:45269)
0806 22:39:28.646749 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:28.646754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:28.646853 (+ 99us) server_negotiation.cc:407] Connection header received
0806 22:39:28.647217 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:28.647222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:28.647323 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:28.647483 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:39:28.647933 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.648747 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.650443 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.650893 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.651014 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:28.651710 (+ 696us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:39:28.652465 (+ 755us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:39:28.652645 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:28.652857 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 22:39:28.652916 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 22:39:28.665134 10435 raft_consensus.cc:1261] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981 [term 1 FOLLOWER]: Refusing update from remote peer a9333fc00894494599ec3b2bcbcd6599: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:28.665134 10248 raft_consensus.cc:1261] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45 [term 1 FOLLOWER]: Refusing update from remote peer a9333fc00894494599ec3b2bcbcd6599: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:28.666574 10541 consensus_queue.cc:1035] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a866f84adbd54df6b7e608c570411981" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 46387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:28.667150 10537 consensus_queue.cc:1035] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b475a67fbe34fe6abaa71810948cc45" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 45555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:28.682706 10546 mvcc.cc:205] Tried to move back new op lower bound from 7315685247636783104 to 7315685247383535616. Current Snapshot: MvccSnapshot[applied={T|T < 7315685247636783104}]
I20260806 22:39:28.699940 9873 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:39:28.700343 9873 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:39:28.705366 9873 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d3c49c158cc4600a2819b381f64fdf5: Sending DeleteTablet for 3 replicas of tablet e9a9beace0ae49eb81b32609e5654bb9
I20260806 22:39:28.707237 10221 tablet_service.cc:1514] Processing DeleteTablet for tablet e9a9beace0ae49eb81b32609e5654bb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:28 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:32960
I20260806 22:39:28.707237 10409 tablet_service.cc:1514] Processing DeleteTablet for tablet e9a9beace0ae49eb81b32609e5654bb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:28 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:34468
I20260806 22:39:28.707348 10040 tablet_service.cc:1514] Processing DeleteTablet for tablet e9a9beace0ae49eb81b32609e5654bb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:28 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:43030
I20260806 22:39:28.710068 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 9954
I20260806 22:39:28.718137 10557 tablet_replica.cc:330] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45: stopping tablet replica
I20260806 22:39:28.718856 10557 raft_consensus.cc:2229] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:39:28.721997 10557 raft_consensus.cc:2258] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:28.729139 10558 tablet_replica.cc:330] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981: stopping tablet replica
I20260806 22:39:28.729835 10558 raft_consensus.cc:2229] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:39:28.730592 10558 raft_consensus.cc:2258] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:28.731674 10560 tablet_replica.cc:330] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599: stopping tablet replica
I20260806 22:39:28.731712 10557 ts_tablet_manager.cc:1905] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:28.732359 10560 raft_consensus.cc:2229] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:39:28.733353 10560 raft_consensus.cc:2258] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:28.737987 10560 ts_tablet_manager.cc:1905] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:28.738914 10558 ts_tablet_manager.cc:1905] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:28.741008 10557 ts_tablet_manager.cc:1918] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:28.741305 10557 log.cc:1196] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/e9a9beace0ae49eb81b32609e5654bb9
I20260806 22:39:28.742024 10557 ts_tablet_manager.cc:1939] T e9a9beace0ae49eb81b32609e5654bb9 P 6b475a67fbe34fe6abaa71810948cc45: Deleting consensus metadata
I20260806 22:39:28.744772 9851 catalog_manager.cc:4841] TS 6b475a67fbe34fe6abaa71810948cc45 (127.15.198.130:45555): tablet e9a9beace0ae49eb81b32609e5654bb9 (table test-table [id=dad3e89f73414adfb3c6a71f2b0a103d]) successfully deleted
Aug 06 22:39:28 dist-test-slave-h3ft krb5kdc[9807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:39:28.747140 10560 ts_tablet_manager.cc:1918] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:28.747400 10560 log.cc:1196] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/e9a9beace0ae49eb81b32609e5654bb9
I20260806 22:39:28.748164 10560 ts_tablet_manager.cc:1939] T e9a9beace0ae49eb81b32609e5654bb9 P a9333fc00894494599ec3b2bcbcd6599: Deleting consensus metadata
I20260806 22:39:28.750638 10558 ts_tablet_manager.cc:1918] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:28.750914 10558 log.cc:1196] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/e9a9beace0ae49eb81b32609e5654bb9
I20260806 22:39:28.750979 9851 catalog_manager.cc:4841] TS a9333fc00894494599ec3b2bcbcd6599 (127.15.198.129:45269): tablet e9a9beace0ae49eb81b32609e5654bb9 (table test-table [id=dad3e89f73414adfb3c6a71f2b0a103d]) successfully deleted
I20260806 22:39:28.751639 10558 ts_tablet_manager.cc:1939] T e9a9beace0ae49eb81b32609e5654bb9 P a866f84adbd54df6b7e608c570411981: Deleting consensus metadata
I20260806 22:39:28.754274 9855 catalog_manager.cc:4841] TS a866f84adbd54df6b7e608c570411981 (127.15.198.131:46387): tablet e9a9beace0ae49eb81b32609e5654bb9 (table test-table [id=dad3e89f73414adfb3c6a71f2b0a103d]) successfully deleted
I20260806 22:39:28.760690 10525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:28.712059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45022 (local address 127.15.198.129:45269)
0806 22:39:28.712370 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:28.712381 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:28.727137 (+ 14756us) server_negotiation.cc:407] Connection header received
0806 22:39:28.727955 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:28.727966 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:28.728127 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:28.728235 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:28.731165 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.732643 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.735183 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.736059 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.736198 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:28.748774 (+ 12576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:28.748791 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:28.748817 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:28.748873 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:28.756636 (+ 7763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:28.757708 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:28.757722 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:28.757726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:28.757968 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:28.758512 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:28.758526 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:28.758530 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:28.759152 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:28.759598 (+ 446us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:28.759867 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 22:39:28.760206 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260806 22:39:28.761734 10121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:28.880178 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 9954
I20260806 22:39:28.914302 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 10135
Aug 06 22:39:28 dist-test-slave-h3ft krb5kdc[9807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.130@KRBTEST.COM
I20260806 22:39:28.933811 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:28.915413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51372 (local address 127.15.198.130:45555)
0806 22:39:28.915630 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:28.915635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:28.915657 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:39:28.915775 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:28.915780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:28.915861 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:28.915989 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:28.917569 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.918429 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.920510 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:28.920959 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:28.921083 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:28.926120 (+ 5037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:28.926143 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:28.926166 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:28.926216 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:28.929824 (+ 3608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:28.930905 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:28.930916 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:28.930919 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:28.931165 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:28.931644 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:28.931649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:28.931651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:28.932227 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:28.932523 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:28.932940 (+ 417us) server_negotiation.cc:299] Negotiation successful
0806 22:39:28.933409 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 22:39:28.934762 10307 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:29.036461 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 10135
I20260806 22:39:29.065096 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 10321
Aug 06 22:39:29 dist-test-slave-h3ft krb5kdc[9807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.131@KRBTEST.COM
I20260806 22:39:29.084818 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:29.066288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59490 (local address 127.15.198.131:46387)
0806 22:39:29.066531 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:29.066536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:29.066558 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:39:29.066644 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:29.066649 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:29.066724 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:29.066821 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:29.068633 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:29.069511 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:29.070931 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:29.071670 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:29.071795 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:29.077633 (+ 5838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:29.077660 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:29.077683 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:29.077745 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:29.081196 (+ 3451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:29.082224 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:29.082235 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:29.082237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:29.082431 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:29.082961 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:29.082965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:29.082967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:29.083548 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:29.083830 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:29.084167 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 22:39:29.084384 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 22:39:29.085793 10496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:29.188367 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 10321
I20260806 22:39:29.218253 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 9828
I20260806 22:39:29.219928 9912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:29.351419 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 9828
2026-08-06T22:39:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5845 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T22:39:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:39:29Z Disabled control of system clock
I20260806 22:39:29.481876 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:43657
--webserver_interface=127.15.198.190
--webserver_port=0
--builtin_ntp_servers=127.15.198.148:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:43657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:39:29.941437 10572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:29.941912 10572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:29.942145 10572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:29.951553 10572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:29.951726 10572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:29.951853 10572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:29.951972 10572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:29.952064 10572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:29.962949 10572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:43657
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:43657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:29.964895 10572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:29.967484 10572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:29.982206 10582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:29.982340 10580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:29.982316 10579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:29.983882 10572 server_base.cc:1034] running on GCE node
I20260806 22:39:29.985003 10572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:29.986871 10572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:29.988116 10572 hybrid_clock.cc:648] HybridClock initialized: now 1786055969988068 us; error 35 us; skew 500 ppm
I20260806 22:39:29.988579 10572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:39:29.991804 10572 webserver.cc:460] Webserver started at http://127.15.198.190:35787/ using document root <none> and password file <none>
I20260806 22:39:29.992769 10572 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:29.992903 10572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:29.993319 10572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:29.997224 10572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "f78a9c86b2214ae681fe0a8988f0a7dd"
format_stamp: "Formatted at 2026-08-06 22:39:29 on dist-test-slave-h3ft"
server_key: "54c66f3d59113d40370246e51af5b16d"
server_key_iv: "04aca897c848a1697ddac1aba5c7edd2"
server_key_version: "encryptionkey@0"
I20260806 22:39:29.998240 10572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "f78a9c86b2214ae681fe0a8988f0a7dd"
format_stamp: "Formatted at 2026-08-06 22:39:29 on dist-test-slave-h3ft"
server_key: "54c66f3d59113d40370246e51af5b16d"
server_key_iv: "04aca897c848a1697ddac1aba5c7edd2"
server_key_version: "encryptionkey@0"
I20260806 22:39:30.005468 10572 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 22:39:30.010766 10592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:30.012794 10572 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:39:30.013074 10572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "f78a9c86b2214ae681fe0a8988f0a7dd"
format_stamp: "Formatted at 2026-08-06 22:39:29 on dist-test-slave-h3ft"
server_key: "54c66f3d59113d40370246e51af5b16d"
server_key_iv: "04aca897c848a1697ddac1aba5c7edd2"
server_key_version: "encryptionkey@0"
I20260806 22:39:30.013391 10572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:30.041457 10572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:30.042711 10572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:30.043145 10572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:30.068470 10572 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:43657
I20260806 22:39:30.068429 10657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:43657 every 8 connection(s)
I20260806 22:39:30.070744 10572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:39:30.076529 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:30.079115 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 10572
I20260806 22:39:30.079351 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:39:30.079921 16154 external_mini_cluster.cc:1428] Setting key 7eec4517733b176a1d286ccf30df9b47
I20260806 22:39:30.094128 10659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd: Bootstrap starting.
I20260806 22:39:30.098001 10666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:30.082651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44370 (local address 127.15.198.190:43657)
0806 22:39:30.084381 (+ 1730us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:30.084396 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:30.084458 (+ 62us) server_negotiation.cc:407] Connection header received
0806 22:39:30.086415 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:30.086448 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:30.087142 (+ 694us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:30.087557 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:30.088361 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:30.089946 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:30.092397 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:30.093227 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:30.093382 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:30.095097 (+ 1715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:30.095112 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:30.095132 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:30.095825 (+ 693us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:30.095923 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:30.095949 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:30.096077 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:30.096325 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 22:39:30.096771 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1029,"thread_start_us":662,"threads_started":1}
I20260806 22:39:30.099458 10659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:30.100978 10659 log.cc:824] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:30.104986 10659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd: No bootstrap required, opened a new log
I20260806 22:39:30.114491 10659 raft_consensus.cc:348] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f78a9c86b2214ae681fe0a8988f0a7dd" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 43657 } }
I20260806 22:39:30.115129 10659 raft_consensus.cc:374] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:30.115267 10659 raft_consensus.cc:729] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f78a9c86b2214ae681fe0a8988f0a7dd, State: Initialized, Role: FOLLOWER
I20260806 22:39:30.116106 10659 consensus_queue.cc:260] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f78a9c86b2214ae681fe0a8988f0a7dd" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 43657 } }
I20260806 22:39:30.116405 10659 raft_consensus.cc:388] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:39:30.116587 10659 raft_consensus.cc:482] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:39:30.116814 10659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:30.119863 10659 raft_consensus.cc:504] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f78a9c86b2214ae681fe0a8988f0a7dd" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 43657 } }
I20260806 22:39:30.120560 10659 leader_election.cc:302] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f78a9c86b2214ae681fe0a8988f0a7dd; no voters:
I20260806 22:39:30.121385 10659 leader_election.cc:288] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:39:30.121572 10669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:30.124202 10659 sys_catalog.cc:564] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:30.126762 10669 raft_consensus.cc:686] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [term 1 LEADER]: Becoming Leader. State: Replica: f78a9c86b2214ae681fe0a8988f0a7dd, State: Running, Role: LEADER
I20260806 22:39:30.128217 10669 consensus_queue.cc:237] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f78a9c86b2214ae681fe0a8988f0a7dd" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 43657 } }
I20260806 22:39:30.132931 10671 sys_catalog.cc:455] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f78a9c86b2214ae681fe0a8988f0a7dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f78a9c86b2214ae681fe0a8988f0a7dd" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 43657 } } }
I20260806 22:39:30.132885 10672 sys_catalog.cc:455] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f78a9c86b2214ae681fe0a8988f0a7dd. Latest consensus state: current_term: 1 leader_uuid: "f78a9c86b2214ae681fe0a8988f0a7dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f78a9c86b2214ae681fe0a8988f0a7dd" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 43657 } } }
I20260806 22:39:30.133528 10671 sys_catalog.cc:458] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:30.133529 10672 sys_catalog.cc:458] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd [sys.catalog]: This master's current role is: LEADER
W20260806 22:39:30.140748 10690 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:39:30.140934 10690 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:39:30.141147 10691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:39:30.143796 10691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:39:30.155912 10691 catalog_manager.cc:1347] Generated new cluster ID: 9ad2514e06374f6bbd09f412489198cd
I20260806 22:39:30.156086 10691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:39:30.174439 10691 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:39:30.176735 10691 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:39:30.196687 10691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd: Generated new TSK 0
I20260806 22:39:30.197948 10691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:39:30.208779 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:43657
--builtin_ntp_servers=127.15.198.148:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:39:30.646736 10698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:30.647197 10698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:30.647307 10698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:30.655412 10698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:30.655623 10698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:30.655906 10698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:39:30.666676 10698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:43657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:30.668524 10698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:30.670854 10698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:30.687527 10703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:30.687500 10704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:30.687407 10706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:30.688915 10698 server_base.cc:1034] running on GCE node
I20260806 22:39:30.689754 10698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:30.691114 10698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:30.692377 10698 hybrid_clock.cc:648] HybridClock initialized: now 1786055970692293 us; error 68 us; skew 500 ppm
I20260806 22:39:30.692822 10698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:39:30.696103 10698 webserver.cc:460] Webserver started at http://127.15.198.129:44251/ using document root <none> and password file <none>
I20260806 22:39:30.697196 10698 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:30.697353 10698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:30.697754 10698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:30.701593 10698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "45b3a50fdd3345c4a4ae521cb8211f82"
format_stamp: "Formatted at 2026-08-06 22:39:30 on dist-test-slave-h3ft"
server_key: "0ba6c1057c0eae4454383fbf232ea0e6"
server_key_iv: "a0a3aacf44b52d1ccad52801831962d9"
server_key_version: "encryptionkey@0"
I20260806 22:39:30.702631 10698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "45b3a50fdd3345c4a4ae521cb8211f82"
format_stamp: "Formatted at 2026-08-06 22:39:30 on dist-test-slave-h3ft"
server_key: "0ba6c1057c0eae4454383fbf232ea0e6"
server_key_iv: "a0a3aacf44b52d1ccad52801831962d9"
server_key_version: "encryptionkey@0"
I20260806 22:39:30.709515 10698 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 22:39:30.714205 10714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:30.716069 10698 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 22:39:30.716305 10698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "45b3a50fdd3345c4a4ae521cb8211f82"
format_stamp: "Formatted at 2026-08-06 22:39:30 on dist-test-slave-h3ft"
server_key: "0ba6c1057c0eae4454383fbf232ea0e6"
server_key_iv: "a0a3aacf44b52d1ccad52801831962d9"
server_key_version: "encryptionkey@0"
I20260806 22:39:30.716609 10698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:30.761813 10698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:30.763399 10698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:30.763850 10698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:30.765422 10698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:30.767396 10698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:30.767542 10698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:30.767697 10698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:30.767804 10698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:30.806252 10698 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:37953
I20260806 22:39:30.806318 10878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:37953 every 8 connection(s)
I20260806 22:39:30.807979 10698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:39:30.813112 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 10698
I20260806 22:39:30.813337 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:39:30.814097 16154 external_mini_cluster.cc:1428] Setting key 218ceb2f5624846e7e12159509048acc
I20260806 22:39:30.817958 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:43657
--builtin_ntp_servers=127.15.198.148:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:39:30.825165 10882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:30.811388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:42873 (local address 127.15.198.190:43657)
0806 22:39:30.812005 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:30.812010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:30.813721 (+ 1711us) server_negotiation.cc:407] Connection header received
0806 22:39:30.816364 (+ 2643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:30.816369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:30.816518 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:30.816675 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:30.818257 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:30.819391 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:30.821567 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:30.822057 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:30.822254 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:30.824056 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:30.824063 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:30.824067 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:30.824094 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:30.824152 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:30.824168 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:30.824301 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:30.824587 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 22:39:30.824717 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":332,"threads_started":1}
I20260806 22:39:30.826606 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:30.812043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:43657 (local address 127.15.198.129:42873)
0806 22:39:30.813455 (+ 1412us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:30.813537 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:30.815973 (+ 2436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:30.817046 (+ 1073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:30.817057 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:30.817170 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:30.817983 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:30.818005 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:30.819560 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:30.819567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:30.821253 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:30.821269 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:30.822326 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:30.822333 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:30.822631 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:30.823578 (+ 947us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:39:30.823581 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:30.823636 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:39:30.823745 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:30.824298 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:30.824312 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:30.824323 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 22:39:30.824566 (+ 243us) client_negotiation.cc:240] Negotiation successful
0806 22:39:30.825137 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":443,"threads_started":1}
I20260806 22:39:30.828809 10879 heartbeater.cc:347] Connected to a master server at 127.15.198.190:43657
I20260806 22:39:30.829300 10879 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:30.830319 10879 heartbeater.cc:511] Master 127.15.198.190:43657 requested a full tablet report, sending...
I20260806 22:39:30.833158 10613 ts_manager.cc:194] Registered new tserver with Master: 45b3a50fdd3345c4a4ae521cb8211f82 (127.15.198.129:37953)
I20260806 22:39:30.834924 10613 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.198.129:42873
W20260806 22:39:31.262182 10885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:31.262588 10885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:31.262687 10885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:31.270869 10885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:31.271129 10885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:31.271330 10885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:39:31.282006 10885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:43657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:31.283979 10885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:31.286366 10885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:31.302191 10893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:31.302130 10897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:31.302167 10894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:31.303267 10885 server_base.cc:1034] running on GCE node
I20260806 22:39:31.304018 10885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:31.305223 10885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:31.306453 10885 hybrid_clock.cc:648] HybridClock initialized: now 1786055971306366 us; error 75 us; skew 500 ppm
I20260806 22:39:31.306849 10885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:39:31.309599 10885 webserver.cc:460] Webserver started at http://127.15.198.130:36781/ using document root <none> and password file <none>
I20260806 22:39:31.310506 10885 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:31.310642 10885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:31.311106 10885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:31.315407 10885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "32cfe5582b0d4a379772b4d1e53b07a7"
format_stamp: "Formatted at 2026-08-06 22:39:31 on dist-test-slave-h3ft"
server_key: "0d1cae5a587909e5e7ef385cc2eabf1e"
server_key_iv: "9fff2d7f4822b8e4b8960f4fff7d9ed6"
server_key_version: "encryptionkey@0"
I20260806 22:39:31.316342 10885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "32cfe5582b0d4a379772b4d1e53b07a7"
format_stamp: "Formatted at 2026-08-06 22:39:31 on dist-test-slave-h3ft"
server_key: "0d1cae5a587909e5e7ef385cc2eabf1e"
server_key_iv: "9fff2d7f4822b8e4b8960f4fff7d9ed6"
server_key_version: "encryptionkey@0"
I20260806 22:39:31.322916 10885 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 22:39:31.327495 10903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:31.329321 10885 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 22:39:31.329530 10885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "32cfe5582b0d4a379772b4d1e53b07a7"
format_stamp: "Formatted at 2026-08-06 22:39:31 on dist-test-slave-h3ft"
server_key: "0d1cae5a587909e5e7ef385cc2eabf1e"
server_key_iv: "9fff2d7f4822b8e4b8960f4fff7d9ed6"
server_key_version: "encryptionkey@0"
I20260806 22:39:31.329790 10885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:31.343880 10885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:31.344929 10885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:31.345294 10885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:31.346522 10885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:31.348366 10885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:31.348493 10885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:31.348656 10885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:31.348748 10885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:31.387136 10885 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:36795
I20260806 22:39:31.387249 11065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:36795 every 8 connection(s)
I20260806 22:39:31.388887 10885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:39:31.397956 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 10885
I20260806 22:39:31.398228 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:39:31.398789 16154 external_mini_cluster.cc:1428] Setting key 27368470725323cfcdc51276e8c09534
I20260806 22:39:31.402818 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:43657
--builtin_ntp_servers=127.15.198.148:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:39:31.406215 11069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:31.392362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:56475 (local address 127.15.198.190:43657)
0806 22:39:31.392964 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:31.392970 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:31.394623 (+ 1653us) server_negotiation.cc:407] Connection header received
0806 22:39:31.396947 (+ 2324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:31.396963 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:31.397091 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:31.397244 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:31.398897 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:31.399880 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:31.402312 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:31.403095 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:31.403252 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:31.405032 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:31.405041 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:31.405049 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:31.405079 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:31.405152 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:31.405174 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:31.405281 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:31.405574 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 22:39:31.405714 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":315,"threads_started":1}
I20260806 22:39:31.407651 11070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:31.393226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:43657 (local address 127.15.198.130:56475)
0806 22:39:31.394385 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:31.394429 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:31.396546 (+ 2117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:31.397631 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:31.397642 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:31.397775 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:31.398679 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:31.398700 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:31.400079 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:31.400084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:31.402140 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:31.402157 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:31.403346 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:31.403351 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:31.403605 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:31.404576 (+ 971us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:39:31.404579 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:31.404645 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:39:31.404763 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:31.405292 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:31.405302 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:31.405310 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:39:31.405529 (+ 219us) client_negotiation.cc:240] Negotiation successful
0806 22:39:31.405977 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":338,"threads_started":1}
I20260806 22:39:31.409607 11066 heartbeater.cc:347] Connected to a master server at 127.15.198.190:43657
I20260806 22:39:31.410039 11066 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:31.411234 11066 heartbeater.cc:511] Master 127.15.198.190:43657 requested a full tablet report, sending...
I20260806 22:39:31.413447 10613 ts_manager.cc:194] Registered new tserver with Master: 32cfe5582b0d4a379772b4d1e53b07a7 (127.15.198.130:36795)
I20260806 22:39:31.414742 10613 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.198.130:56475
I20260806 22:39:31.839655 10879 heartbeater.cc:503] Master 127.15.198.190:43657 was elected leader, sending a full tablet report...
W20260806 22:39:31.839499 11071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:31.839833 11071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:31.840022 11071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:31.848681 11071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:31.848812 11071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:31.849009 11071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:39:31.859220 11071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:43657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:31.861083 11071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:31.863509 11071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:31.880182 11079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:31.880182 11078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:31.880442 11081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:31.881268 11071 server_base.cc:1034] running on GCE node
I20260806 22:39:31.882129 11071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:31.883447 11071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:31.884716 11071 hybrid_clock.cc:648] HybridClock initialized: now 1786055971884665 us; error 50 us; skew 500 ppm
I20260806 22:39:31.885133 11071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:39:31.888257 11071 webserver.cc:460] Webserver started at http://127.15.198.131:32875/ using document root <none> and password file <none>
I20260806 22:39:31.889147 11071 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:31.889295 11071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:31.889698 11071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:31.893355 11071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "7c16c395ee5b425db2d1f44309cbe320"
format_stamp: "Formatted at 2026-08-06 22:39:31 on dist-test-slave-h3ft"
server_key: "f56651f26f8d49e00505c1a45e86cc1d"
server_key_iv: "f0e5ccc7b36d2791f92be793f615f0be"
server_key_version: "encryptionkey@0"
I20260806 22:39:31.894336 11071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "7c16c395ee5b425db2d1f44309cbe320"
format_stamp: "Formatted at 2026-08-06 22:39:31 on dist-test-slave-h3ft"
server_key: "f56651f26f8d49e00505c1a45e86cc1d"
server_key_iv: "f0e5ccc7b36d2791f92be793f615f0be"
server_key_version: "encryptionkey@0"
I20260806 22:39:31.901177 11071 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.003s
I20260806 22:39:31.905800 11093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:31.907853 11071 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:39:31.908085 11071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "7c16c395ee5b425db2d1f44309cbe320"
format_stamp: "Formatted at 2026-08-06 22:39:31 on dist-test-slave-h3ft"
server_key: "f56651f26f8d49e00505c1a45e86cc1d"
server_key_iv: "f0e5ccc7b36d2791f92be793f615f0be"
server_key_version: "encryptionkey@0"
I20260806 22:39:31.908391 11071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:31.925056 11071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:31.926209 11071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:31.926595 11071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:31.928001 11071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:31.930083 11071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:31.930330 11071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:31.930527 11071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:31.930634 11071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:31.970299 11071 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:42521
I20260806 22:39:31.970341 11259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:42521 every 8 connection(s)
I20260806 22:39:31.972160 11071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:39:31.980080 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 11071
I20260806 22:39:31.980343 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:39:31.980907 16154 external_mini_cluster.cc:1428] Setting key df4c7bd845a763ca2f2feb8e74ace637
I20260806 22:39:31.988576 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:31.975540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:41951 (local address 127.15.198.190:43657)
0806 22:39:31.976112 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:31.976120 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:31.977753 (+ 1633us) server_negotiation.cc:407] Connection header received
0806 22:39:31.980085 (+ 2332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:31.980090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:31.980182 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:31.980339 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:31.982031 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:31.983099 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:31.985269 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:31.985755 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:31.985866 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:31.987689 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:31.987694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:31.987698 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:31.987718 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:31.987770 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:31.987784 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:31.987862 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:31.988099 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 22:39:31.988208 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":252,"threads_started":1}
I20260806 22:39:31.989892 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:31.976335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:43657 (local address 127.15.198.131:41951)
0806 22:39:31.977550 (+ 1215us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:31.977597 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:31.979689 (+ 2092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:31.980656 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:31.980666 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:31.980774 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:31.981832 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:31.981855 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:31.983290 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:31.983299 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:31.985104 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:31.985118 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:31.986035 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:31.986041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:31.986303 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:31.987317 (+ 1014us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:39:31.987319 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:31.987371 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:39:31.987475 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:31.987874 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:31.987883 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:31.987890 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:39:31.988068 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 22:39:31.988457 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":365,"threads_started":1}
I20260806 22:39:31.991693 11261 heartbeater.cc:347] Connected to a master server at 127.15.198.190:43657
I20260806 22:39:31.992121 11261 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:31.993193 11261 heartbeater.cc:511] Master 127.15.198.190:43657 requested a full tablet report, sending...
I20260806 22:39:31.995198 10613 ts_manager.cc:194] Registered new tserver with Master: 7c16c395ee5b425db2d1f44309cbe320 (127.15.198.131:42521)
I20260806 22:39:31.996402 10613 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.198.131:41951
I20260806 22:39:31.997632 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:39:32.009397 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.002726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44374 (local address 127.15.198.190:43657)
0806 22:39:32.002951 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:32.002956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:32.003556 (+ 600us) server_negotiation.cc:407] Connection header received
0806 22:39:32.003700 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:32.003704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:32.003780 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:32.003880 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:32.004412 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.005233 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.006676 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.007247 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.007394 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:32.008658 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:32.008663 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:32.008667 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:32.008687 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:32.008727 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:32.008740 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:32.008868 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:32.009063 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 22:39:32.009118 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 22:39:32.014468 10613 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:39:32.023044 10613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:39:32.063046 11278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.039351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:42521 (local address 127.0.0.1:41778)
0806 22:39:32.040474 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:32.040538 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:32.040830 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:32.054230 (+ 13400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:32.054235 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:32.054264 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:32.054978 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:32.054994 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.057282 (+ 2288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.057289 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:32.059614 (+ 2325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:32.059629 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.061793 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.061800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:32.062049 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:32.062141 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 22:39:32.062265 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 22:39:32.062486 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":133,"thread_start_us":15070,"threads_started":1}
I20260806 22:39:32.063506 11285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.039391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:36795 (local address 127.0.0.1:44936)
0806 22:39:32.052075 (+ 12684us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:32.052107 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:32.052350 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:32.054019 (+ 1669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:32.054030 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:32.054071 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:32.055428 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:32.055441 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.057493 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.057499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:32.060269 (+ 2770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:32.060287 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.062641 (+ 2354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.062648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:32.062804 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:32.062836 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 22:39:32.062930 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 22:39:32.063128 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12477,"mutex_wait_us":196,"thread_start_us":518,"threads_started":1}
I20260806 22:39:32.064284 11286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.041694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50714 (local address 127.15.198.129:37953)
0806 22:39:32.051069 (+ 9375us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:32.051085 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:32.054967 (+ 3882us) server_negotiation.cc:407] Connection header received
0806 22:39:32.055274 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:32.055284 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:32.055589 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:32.055816 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:32.056465 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.058169 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.060917 (+ 2748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.062420 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.062638 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:32.062804 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:32.063638 (+ 834us) server_negotiation.cc:299] Negotiation successful
0806 22:39:32.063865 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9013,"thread_start_us":9381,"threads_started":1}
I20260806 22:39:32.064383 11283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.039833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41778 (local address 127.15.198.131:42521)
0806 22:39:32.053558 (+ 13725us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:32.053567 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:32.053608 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:39:32.053719 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:32.053729 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:32.053980 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:32.054437 (+ 457us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:32.055371 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.057117 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.059825 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.061539 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.063450 (+ 1911us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:32.063708 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:32.063799 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 22:39:32.064059 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13515,"thread_start_us":13942,"threads_started":1}
I20260806 22:39:32.064670 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.039962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44936 (local address 127.15.198.130:36795)
0806 22:39:32.053026 (+ 13064us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:32.053041 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:32.053094 (+ 53us) server_negotiation.cc:407] Connection header received
0806 22:39:32.053250 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:32.053263 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:32.053683 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:32.055226 (+ 1543us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:32.055592 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.057297 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.060534 (+ 3237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.062038 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.062224 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:32.062501 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:32.063066 (+ 565us) server_negotiation.cc:299] Negotiation successful
0806 22:39:32.064111 (+ 1045us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12865,"thread_start_us":12736,"threads_started":1}
I20260806 22:39:32.066143 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.041170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:37953 (local address 127.0.0.1:50714)
0806 22:39:32.054774 (+ 13604us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:32.054793 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:32.055000 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:32.055816 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:32.055822 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:32.055849 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:32.056241 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:32.056257 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.058375 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.058380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:32.060731 (+ 2351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:32.060746 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.062972 (+ 2226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.062979 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:32.063215 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:32.063264 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 22:39:32.063354 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 22:39:32.063500 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13454,"mutex_wait_us":1,"thread_start_us":10480,"threads_started":1}
I20260806 22:39:32.069447 11163 tablet_service.cc:1467] Processing CreateTablet for tablet eb8aac70d03a4fad9b74ba834de1fc43 (DEFAULT_TABLE table=test-table [id=99f2a868424d47769329137956346daf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:32.069451 10975 tablet_service.cc:1467] Processing CreateTablet for tablet eb8aac70d03a4fad9b74ba834de1fc43 (DEFAULT_TABLE table=test-table [id=99f2a868424d47769329137956346daf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:32.070334 10788 tablet_service.cc:1467] Processing CreateTablet for tablet eb8aac70d03a4fad9b74ba834de1fc43 (DEFAULT_TABLE table=test-table [id=99f2a868424d47769329137956346daf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:32.071286 11163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb8aac70d03a4fad9b74ba834de1fc43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:32.071290 10975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb8aac70d03a4fad9b74ba834de1fc43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:32.072695 10788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb8aac70d03a4fad9b74ba834de1fc43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:32.094852 11289 tablet_bootstrap.cc:492] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7: Bootstrap starting.
I20260806 22:39:32.094856 11288 tablet_bootstrap.cc:492] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320: Bootstrap starting.
I20260806 22:39:32.101405 11287 tablet_bootstrap.cc:492] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82: Bootstrap starting.
I20260806 22:39:32.102741 11288 tablet_bootstrap.cc:654] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:32.104077 11289 tablet_bootstrap.cc:654] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:32.104229 11288 log.cc:824] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:32.105528 11289 log.cc:824] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:32.105988 11287 tablet_bootstrap.cc:654] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:32.107702 11287 log.cc:824] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:32.109532 11288 tablet_bootstrap.cc:492] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320: No bootstrap required, opened a new log
I20260806 22:39:32.109887 11288 ts_tablet_manager.cc:1397] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.006s
I20260806 22:39:32.111613 11287 tablet_bootstrap.cc:492] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82: No bootstrap required, opened a new log
I20260806 22:39:32.111657 11289 tablet_bootstrap.cc:492] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7: No bootstrap required, opened a new log
I20260806 22:39:32.111943 11289 ts_tablet_manager.cc:1397] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.006s
I20260806 22:39:32.111944 11287 ts_tablet_manager.cc:1397] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260806 22:39:32.119434 11288 raft_consensus.cc:348] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c16c395ee5b425db2d1f44309cbe320" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 42521 } } peers { permanent_uuid: "32cfe5582b0d4a379772b4d1e53b07a7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36795 } } peers { permanent_uuid: "45b3a50fdd3345c4a4ae521cb8211f82" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37953 } }
I20260806 22:39:32.120085 11288 raft_consensus.cc:374] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:32.120206 11288 raft_consensus.cc:729] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c16c395ee5b425db2d1f44309cbe320, State: Initialized, Role: FOLLOWER
I20260806 22:39:32.121002 11288 consensus_queue.cc:260] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c16c395ee5b425db2d1f44309cbe320" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 42521 } } peers { permanent_uuid: "32cfe5582b0d4a379772b4d1e53b07a7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36795 } } peers { permanent_uuid: "45b3a50fdd3345c4a4ae521cb8211f82" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37953 } }
I20260806 22:39:32.123288 11261 heartbeater.cc:503] Master 127.15.198.190:43657 was elected leader, sending a full tablet report...
I20260806 22:39:32.124190 11288 ts_tablet_manager.cc:1428] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320: Time spent starting tablet: real 0.014s user 0.013s sys 0.002s
I20260806 22:39:32.124017 11287 raft_consensus.cc:348] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c16c395ee5b425db2d1f44309cbe320" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 42521 } } peers { permanent_uuid: "32cfe5582b0d4a379772b4d1e53b07a7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36795 } } peers { permanent_uuid: "45b3a50fdd3345c4a4ae521cb8211f82" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37953 } }
I20260806 22:39:32.124006 11289 raft_consensus.cc:348] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c16c395ee5b425db2d1f44309cbe320" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 42521 } } peers { permanent_uuid: "32cfe5582b0d4a379772b4d1e53b07a7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36795 } } peers { permanent_uuid: "45b3a50fdd3345c4a4ae521cb8211f82" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37953 } }
I20260806 22:39:32.124634 11289 raft_consensus.cc:374] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:32.124634 11287 raft_consensus.cc:374] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:32.124743 11287 raft_consensus.cc:729] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45b3a50fdd3345c4a4ae521cb8211f82, State: Initialized, Role: FOLLOWER
I20260806 22:39:32.124743 11289 raft_consensus.cc:729] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32cfe5582b0d4a379772b4d1e53b07a7, State: Initialized, Role: FOLLOWER
I20260806 22:39:32.125554 11287 consensus_queue.cc:260] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c16c395ee5b425db2d1f44309cbe320" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 42521 } } peers { permanent_uuid: "32cfe5582b0d4a379772b4d1e53b07a7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36795 } } peers { permanent_uuid: "45b3a50fdd3345c4a4ae521cb8211f82" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37953 } }
I20260806 22:39:32.125562 11289 consensus_queue.cc:260] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c16c395ee5b425db2d1f44309cbe320" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 42521 } } peers { permanent_uuid: "32cfe5582b0d4a379772b4d1e53b07a7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36795 } } peers { permanent_uuid: "45b3a50fdd3345c4a4ae521cb8211f82" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37953 } }
I20260806 22:39:32.127308 11066 heartbeater.cc:503] Master 127.15.198.190:43657 was elected leader, sending a full tablet report...
I20260806 22:39:32.127986 11289 ts_tablet_manager.cc:1428] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
I20260806 22:39:32.128283 11287 ts_tablet_manager.cc:1428] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
W20260806 22:39:32.141530 11068 tablet.cc:2369] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:39:32.224737 11263 tablet.cc:2369] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:39:32.241003 11296 raft_consensus.cc:482] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:39:32.241477 11296 raft_consensus.cc:504] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c16c395ee5b425db2d1f44309cbe320" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 42521 } } peers { permanent_uuid: "32cfe5582b0d4a379772b4d1e53b07a7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36795 } } peers { permanent_uuid: "45b3a50fdd3345c4a4ae521cb8211f82" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37953 } }
I20260806 22:39:32.243867 11296 leader_election.cc:288] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c16c395ee5b425db2d1f44309cbe320 (127.15.198.131:42521), 32cfe5582b0d4a379772b4d1e53b07a7 (127.15.198.130:36795)
I20260806 22:39:32.251559 11283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.244496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:45799 (local address 127.15.198.131:42521)
0806 22:39:32.245070 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:32.245081 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:32.245326 (+ 245us) server_negotiation.cc:407] Connection header received
0806 22:39:32.245634 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:32.245639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:32.245712 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:32.245820 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:32.246470 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.247375 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.249532 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.250432 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.250636 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:32.250727 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:32.251092 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 22:39:32.251257 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341}
I20260806 22:39:32.251606 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.244718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:49213 (local address 127.15.198.130:36795)
0806 22:39:32.245261 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:32.245269 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:32.245964 (+ 695us) server_negotiation.cc:407] Connection header received
0806 22:39:32.246190 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:32.246195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:32.246278 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:32.246390 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:32.247039 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.247755 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.249704 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.250541 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.250754 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:32.250830 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:32.251168 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 22:39:32.251316 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365}
I20260806 22:39:32.251665 11297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.244216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:42521 (local address 127.15.198.129:45799)
0806 22:39:32.245188 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:32.245213 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:32.245439 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:32.245852 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:32.245856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:32.245885 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:32.246311 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:32.246321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.247526 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.247537 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:32.249309 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:32.249324 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.250655 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.250662 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:32.250841 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:32.250892 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 22:39:32.251027 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 22:39:32.251179 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":68,"spinlock_wait_cycles":24832,"thread_start_us":571,"threads_started":1}
I20260806 22:39:32.252604 11298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.244367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:36795 (local address 127.15.198.129:49213)
0806 22:39:32.245827 (+ 1460us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:32.245843 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:32.246015 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:32.246403 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:32.246407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:32.246431 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:32.246818 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:32.246828 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.247909 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.247916 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:32.249489 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:32.249503 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.250754 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.250761 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:32.250932 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:32.250958 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 22:39:32.252062 (+ 1104us) client_negotiation.cc:240] Negotiation successful
0806 22:39:32.252210 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1340,"mutex_wait_us":20,"thread_start_us":989,"threads_started":1}
I20260806 22:39:32.253226 11194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb8aac70d03a4fad9b74ba834de1fc43" candidate_uuid: "45b3a50fdd3345c4a4ae521cb8211f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c16c395ee5b425db2d1f44309cbe320" is_pre_election: true
I20260806 22:39:32.254097 11194 raft_consensus.cc:2454] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45b3a50fdd3345c4a4ae521cb8211f82 in term 0.
I20260806 22:39:32.254021 11002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb8aac70d03a4fad9b74ba834de1fc43" candidate_uuid: "45b3a50fdd3345c4a4ae521cb8211f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32cfe5582b0d4a379772b4d1e53b07a7" is_pre_election: true
I20260806 22:39:32.254921 11002 raft_consensus.cc:2454] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45b3a50fdd3345c4a4ae521cb8211f82 in term 0.
I20260806 22:39:32.255487 10719 leader_election.cc:302] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45b3a50fdd3345c4a4ae521cb8211f82, 7c16c395ee5b425db2d1f44309cbe320; no voters:
I20260806 22:39:32.256093 11296 raft_consensus.cc:2781] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:39:32.256378 11296 raft_consensus.cc:482] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:39:32.256479 11296 raft_consensus.cc:3037] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:32.259553 11296 raft_consensus.cc:504] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c16c395ee5b425db2d1f44309cbe320" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 42521 } } peers { permanent_uuid: "32cfe5582b0d4a379772b4d1e53b07a7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36795 } } peers { permanent_uuid: "45b3a50fdd3345c4a4ae521cb8211f82" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37953 } }
I20260806 22:39:32.260385 11296 leader_election.cc:288] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [CANDIDATE]: Term 1 election: Requested vote from peers 7c16c395ee5b425db2d1f44309cbe320 (127.15.198.131:42521), 32cfe5582b0d4a379772b4d1e53b07a7 (127.15.198.130:36795)
I20260806 22:39:32.261180 11002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb8aac70d03a4fad9b74ba834de1fc43" candidate_uuid: "45b3a50fdd3345c4a4ae521cb8211f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32cfe5582b0d4a379772b4d1e53b07a7"
I20260806 22:39:32.261183 11194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb8aac70d03a4fad9b74ba834de1fc43" candidate_uuid: "45b3a50fdd3345c4a4ae521cb8211f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c16c395ee5b425db2d1f44309cbe320"
I20260806 22:39:32.261485 11002 raft_consensus.cc:3037] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:32.261502 11194 raft_consensus.cc:3037] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:32.264331 11194 raft_consensus.cc:2454] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45b3a50fdd3345c4a4ae521cb8211f82 in term 1.
I20260806 22:39:32.264354 11002 raft_consensus.cc:2454] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45b3a50fdd3345c4a4ae521cb8211f82 in term 1.
I20260806 22:39:32.265039 10715 leader_election.cc:302] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 32cfe5582b0d4a379772b4d1e53b07a7, 45b3a50fdd3345c4a4ae521cb8211f82, 7c16c395ee5b425db2d1f44309cbe320; no voters:
I20260806 22:39:32.265523 11296 raft_consensus.cc:2781] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:32.266134 11296 raft_consensus.cc:686] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [term 1 LEADER]: Becoming Leader. State: Replica: 45b3a50fdd3345c4a4ae521cb8211f82, State: Running, Role: LEADER
I20260806 22:39:32.266866 11296 consensus_queue.cc:237] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c16c395ee5b425db2d1f44309cbe320" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 42521 } } peers { permanent_uuid: "32cfe5582b0d4a379772b4d1e53b07a7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36795 } } peers { permanent_uuid: "45b3a50fdd3345c4a4ae521cb8211f82" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37953 } }
I20260806 22:39:32.273478 10610 catalog_manager.cc:5495] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45b3a50fdd3345c4a4ae521cb8211f82 (127.15.198.129). New cstate: current_term: 1 leader_uuid: "45b3a50fdd3345c4a4ae521cb8211f82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c16c395ee5b425db2d1f44309cbe320" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 42521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32cfe5582b0d4a379772b4d1e53b07a7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45b3a50fdd3345c4a4ae521cb8211f82" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 37953 } health_report { overall_health: HEALTHY } } }
W20260806 22:39:32.311335 10881 tablet.cc:2369] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:39:32.317342 11286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.310129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50724 (local address 127.15.198.129:37953)
0806 22:39:32.310408 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:32.310415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:32.310437 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:39:32.310640 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:32.310645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:32.310770 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:32.310921 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:39:32.311589 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.312398 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.314488 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.314968 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.315156 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:32.315797 (+ 641us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:39:32.316490 (+ 693us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:39:32.316655 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:32.316847 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 22:39:32.316897 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 22:39:32.329087 11194 raft_consensus.cc:1261] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320 [term 1 FOLLOWER]: Refusing update from remote peer 45b3a50fdd3345c4a4ae521cb8211f82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:32.329085 11002 raft_consensus.cc:1261] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7 [term 1 FOLLOWER]: Refusing update from remote peer 45b3a50fdd3345c4a4ae521cb8211f82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:32.330709 11296 consensus_queue.cc:1035] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c16c395ee5b425db2d1f44309cbe320" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 42521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:32.331305 11299 consensus_queue.cc:1035] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32cfe5582b0d4a379772b4d1e53b07a7" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 36795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:32.344864 11302 mvcc.cc:205] Tried to move back new op lower bound from 7315685262645956608 to 7315685262410465280. Current Snapshot: MvccSnapshot[applied={T|T < 7315685262645956608}]
I20260806 22:39:32.367779 10612 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:39:32.368184 10612 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:39:32.372881 10612 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f78a9c86b2214ae681fe0a8988f0a7dd: Sending DeleteTablet for 3 replicas of tablet eb8aac70d03a4fad9b74ba834de1fc43
I20260806 22:39:32.374475 11163 tablet_service.cc:1514] Processing DeleteTablet for tablet eb8aac70d03a4fad9b74ba834de1fc43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:32 UTC) from {username='slave'} at 127.0.0.1:41778
I20260806 22:39:32.374550 10975 tablet_service.cc:1514] Processing DeleteTablet for tablet eb8aac70d03a4fad9b74ba834de1fc43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:32 UTC) from {username='slave'} at 127.0.0.1:44936
I20260806 22:39:32.374523 10788 tablet_service.cc:1514] Processing DeleteTablet for tablet eb8aac70d03a4fad9b74ba834de1fc43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:32 UTC) from {username='slave'} at 127.0.0.1:50714
I20260806 22:39:32.376969 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 10698
I20260806 22:39:32.395818 11323 tablet_replica.cc:330] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320: stopping tablet replica
I20260806 22:39:32.396561 11323 raft_consensus.cc:2229] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:39:32.397303 11323 raft_consensus.cc:2258] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:32.397814 11322 tablet_replica.cc:330] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82: stopping tablet replica
I20260806 22:39:32.398469 11322 raft_consensus.cc:2229] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:39:32.399561 11322 raft_consensus.cc:2258] T eb8aac70d03a4fad9b74ba834de1fc43 P 45b3a50fdd3345c4a4ae521cb8211f82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:32.401120 11323 ts_tablet_manager.cc:1905] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:32.403286 11286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.378606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50726 (local address 127.15.198.129:37953)
0806 22:39:32.378843 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:32.378853 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:32.379217 (+ 364us) server_negotiation.cc:407] Connection header received
0806 22:39:32.379593 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:32.379602 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:32.379731 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:32.379814 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:32.380589 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.382324 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.384839 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.385841 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.385976 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:32.400435 (+ 14459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:32.400450 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:32.400471 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:32.401473 (+ 1002us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:32.401571 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:32.401609 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:32.401717 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:32.402317 (+ 600us) server_negotiation.cc:299] Negotiation successful
0806 22:39:32.402694 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:39:32.404974 10872 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:32.406172 11321 tablet_replica.cc:330] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7: stopping tablet replica
I20260806 22:39:32.406709 11321 raft_consensus.cc:2229] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:39:32.407444 11321 raft_consensus.cc:2258] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:32.410807 11321 ts_tablet_manager.cc:1905] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:32.419020 11321 ts_tablet_manager.cc:1918] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:32.419112 11323 ts_tablet_manager.cc:1918] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:32.419255 11321 log.cc:1196] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/eb8aac70d03a4fad9b74ba834de1fc43
I20260806 22:39:32.419348 11323 log.cc:1196] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/eb8aac70d03a4fad9b74ba834de1fc43
I20260806 22:39:32.419988 11321 ts_tablet_manager.cc:1939] T eb8aac70d03a4fad9b74ba834de1fc43 P 32cfe5582b0d4a379772b4d1e53b07a7: Deleting consensus metadata
I20260806 22:39:32.419998 11323 ts_tablet_manager.cc:1939] T eb8aac70d03a4fad9b74ba834de1fc43 P 7c16c395ee5b425db2d1f44309cbe320: Deleting consensus metadata
I20260806 22:39:32.422361 10596 catalog_manager.cc:4841] TS 7c16c395ee5b425db2d1f44309cbe320 (127.15.198.131:42521): tablet eb8aac70d03a4fad9b74ba834de1fc43 (table test-table [id=99f2a868424d47769329137956346daf]) successfully deleted
I20260806 22:39:32.422533 10593 catalog_manager.cc:4841] TS 32cfe5582b0d4a379772b4d1e53b07a7 (127.15.198.130:36795): tablet eb8aac70d03a4fad9b74ba834de1fc43 (table test-table [id=99f2a868424d47769329137956346daf]) successfully deleted
I20260806 22:39:32.520805 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 10698
I20260806 22:39:32.554283 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 10885
W20260806 22:39:32.554183 10593 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.198.129:37953 (error 108)
I20260806 22:39:32.556360 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.555842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:37953 (local address 127.0.0.1:50736)
0806 22:39:32.556034 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:32.556170 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:37953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 22:39:32.556671 10593 catalog_manager.cc:4584] TS 45b3a50fdd3345c4a4ae521cb8211f82 (127.15.198.129:37953): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eb8aac70d03a4fad9b74ba834de1fc43: Network error: Client connection negotiation failed: client connection to 127.15.198.129:37953: connect: Connection refused (error 111)
I20260806 22:39:32.562770 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.555360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44938 (local address 127.15.198.130:36795)
0806 22:39:32.555584 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:32.555591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:32.555706 (+ 115us) server_negotiation.cc:407] Connection header received
0806 22:39:32.555967 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:32.555973 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:32.556117 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:32.556248 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:32.556922 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.557761 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.559281 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.559723 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.559873 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:32.561135 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:32.561145 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:32.561164 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:32.561912 (+ 748us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:32.561971 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:32.561993 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:32.562149 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:32.562246 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 22:39:32.562371 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 22:39:32.563673 11061 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:32.610690 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.610052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:37953 (local address 127.0.0.1:50744)
0806 22:39:32.610355 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:32.610488 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:37953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260806 22:39:32.612159 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.611661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:37953 (local address 127.0.0.1:50750)
0806 22:39:32.611859 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:32.611960 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:37953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 22:39:32.662724 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 10885
I20260806 22:39:32.673331 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.672686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:37953 (local address 127.0.0.1:50756)
0806 22:39:32.672958 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:32.673083 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:37953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260806 22:39:32.674741 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.674226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:37953 (local address 127.0.0.1:50768)
0806 22:39:32.674431 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:32.674558 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:37953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 22:39:32.692757 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 11071
I20260806 22:39:32.700775 11283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.693773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41784 (local address 127.15.198.131:42521)
0806 22:39:32.694064 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:32.694072 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:32.694098 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:39:32.694246 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:32.694251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:32.694362 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:32.694490 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:32.694961 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.695837 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.697216 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:32.697653 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:32.697797 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:32.699107 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:32.699119 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:32.699140 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:32.699877 (+ 737us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:32.699939 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:32.699962 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:32.700111 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:32.700242 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 22:39:32.700380 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 22:39:32.701651 11251 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:32.755929 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.755238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:37953 (local address 127.0.0.1:50784)
0806 22:39:32.755518 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:32.755714 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:37953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260806 22:39:32.757288 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.756808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:37953 (local address 127.0.0.1:50798)
0806 22:39:32.757069 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:32.757154 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:37953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260806 22:39:32.804191 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 11071
I20260806 22:39:32.833141 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 10572
I20260806 22:39:32.834648 10651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:32.969964 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:32.968582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:37953 (local address 127.0.0.1:50814)
0806 22:39:32.969369 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:32.969538 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.129:37953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":578}
I20260806 22:39:32.970345 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 10572
2026-08-06T22:39:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3616 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:39:33 dist-test-slave-h3ft krb5kdc[11337](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:39:33 dist-test-slave-h3ft krb5kdc[11337](info): set up 2 sockets
Aug 06 22:39:33 dist-test-slave-h3ft krb5kdc[11337](info): commencing operation
krb5kdc: starting...
W20260806 22:39:34.363591 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.333s user 0.009s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:39:34 dist-test-slave-h3ft krb5kdc[11337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:39:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:39:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:39:34.783280 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:33971
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:33305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:33971
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:35.232843 11356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:35.233273 11356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:35.233381 11356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:35.241822 11356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:35.241976 11356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:35.242028 11356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:35.242092 11356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:35.242141 11356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:35.253867 11356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:33305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:33971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:33971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:35.255760 11356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:35.257596 11356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:35.272109 11363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:35.272073 11372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:35.272148 11368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:35.273047 11356 server_base.cc:1034] running on GCE node
I20260806 22:39:35.274117 11356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:35.276005 11356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:35.277254 11356 hybrid_clock.cc:648] HybridClock initialized: now 1786055975277200 us; error 35 us; skew 500 ppm
I20260806 22:39:35.277690 11356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:35 dist-test-slave-h3ft krb5kdc[11337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055975, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:35.285787 11356 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:39:35.288005 11356 webserver.cc:460] Webserver started at http://127.15.198.190:46845/ using document root <none> and password file <none>
I20260806 22:39:35.288967 11356 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:35.289096 11356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:35.289438 11356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:35.293555 11356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "4ab4083310d84446ae9b6452a6253f1c"
format_stamp: "Formatted at 2026-08-06 22:39:35 on dist-test-slave-h3ft"
server_key: "a650ced39337fdc22958e57f4ea6fc93"
server_key_iv: "cf87571444e923dcc80e495b975ed936"
server_key_version: "encryptionkey@0"
I20260806 22:39:35.294514 11356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "4ab4083310d84446ae9b6452a6253f1c"
format_stamp: "Formatted at 2026-08-06 22:39:35 on dist-test-slave-h3ft"
server_key: "a650ced39337fdc22958e57f4ea6fc93"
server_key_iv: "cf87571444e923dcc80e495b975ed936"
server_key_version: "encryptionkey@0"
I20260806 22:39:35.301420 11356 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260806 22:39:35.306232 11382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:35.308111 11356 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260806 22:39:35.308302 11356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "4ab4083310d84446ae9b6452a6253f1c"
format_stamp: "Formatted at 2026-08-06 22:39:35 on dist-test-slave-h3ft"
server_key: "a650ced39337fdc22958e57f4ea6fc93"
server_key_iv: "cf87571444e923dcc80e495b975ed936"
server_key_version: "encryptionkey@0"
I20260806 22:39:35.308517 11356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:35.321393 11356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:35.326599 11356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:35.326905 11356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:35.348728 11356 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:33971
I20260806 22:39:35.348750 11456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:33971 every 8 connection(s)
I20260806 22:39:35.350543 11356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:39:35.355393 11457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:35.358670 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 11356
I20260806 22:39:35.358914 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:39:35.359455 16154 external_mini_cluster.cc:1428] Setting key 8c7ae4f9b91dd7e80372cf55648cd6b9
I20260806 22:39:35.369513 11457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c: Bootstrap starting.
I20260806 22:39:35.374221 11457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:35.375873 11457 log.cc:824] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c: Log is configured to *not* fsync() on all Append() calls
Aug 06 22:39:35 dist-test-slave-h3ft krb5kdc[11337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:39:35.379675 11457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c: No bootstrap required, opened a new log
I20260806 22:39:35.386886 11461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:35.361829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34620 (local address 127.15.198.190:33971)
0806 22:39:35.362728 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:35.362742 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:35.362796 (+ 54us) server_negotiation.cc:407] Connection header received
0806 22:39:35.364752 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:35.364778 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:35.365256 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:35.365649 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:35.367785 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:35.369275 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:35.371197 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:35.371941 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:35.372112 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:35.377936 (+ 5824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:35.377945 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:35.377964 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:35.378012 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:35.381594 (+ 3582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:35.382597 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:35.382605 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:35.382613 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:35.382837 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:35.383355 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:35.383360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:35.383362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:35.384028 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:35.384424 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:35.384959 (+ 535us) server_negotiation.cc:299] Negotiation successful
0806 22:39:35.385530 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":404,"threads_started":1}
I20260806 22:39:35.389026 11457 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4083310d84446ae9b6452a6253f1c" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33971 } }
I20260806 22:39:35.389605 11457 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:35.389739 11457 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ab4083310d84446ae9b6452a6253f1c, State: Initialized, Role: FOLLOWER
I20260806 22:39:35.390535 11457 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4083310d84446ae9b6452a6253f1c" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33971 } }
I20260806 22:39:35.390897 11457 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:39:35.391126 11457 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:39:35.391283 11457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:35.393757 11457 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4083310d84446ae9b6452a6253f1c" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33971 } }
I20260806 22:39:35.394367 11457 leader_election.cc:302] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ab4083310d84446ae9b6452a6253f1c; no voters:
I20260806 22:39:35.395087 11457 leader_election.cc:288] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:39:35.395284 11464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:35.396875 11464 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [term 1 LEADER]: Becoming Leader. State: Replica: 4ab4083310d84446ae9b6452a6253f1c, State: Running, Role: LEADER
I20260806 22:39:35.397528 11464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4083310d84446ae9b6452a6253f1c" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33971 } }
I20260806 22:39:35.398098 11457 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:35.400981 11466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ab4083310d84446ae9b6452a6253f1c. Latest consensus state: current_term: 1 leader_uuid: "4ab4083310d84446ae9b6452a6253f1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4083310d84446ae9b6452a6253f1c" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33971 } } }
I20260806 22:39:35.401149 11465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ab4083310d84446ae9b6452a6253f1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab4083310d84446ae9b6452a6253f1c" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33971 } } }
I20260806 22:39:35.401507 11466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:35.401546 11465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:35.402853 11473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:39:35.410811 11473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:39:35.424098 11473 catalog_manager.cc:1347] Generated new cluster ID: 1fc8a640ea18437aadd8f7efe6d27c37
I20260806 22:39:35.424273 11473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:39:35.450932 11473 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:39:35.453408 11473 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:39:35.467876 11473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c: Generated new TSK 0
I20260806 22:39:35.469334 11473 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.198.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
Entry for principal kudu/127.15.198.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab.
I20260806 22:39:35.567364 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33971
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:33305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:36.018939 11495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:36.019359 11495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:36.019477 11495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:36.027608 11495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:36.027765 11495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:36.027895 11495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:39:36.039299 11495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:33305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.129.keytab
--principal=kudu/127.15.198.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:36.041163 11495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:36.042888 11495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:36.059259 11503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:36.059280 11507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:36.059653 11505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:36.062377 11495 server_base.cc:1034] running on GCE node
I20260806 22:39:36.063436 11495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:36.064726 11495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:36.066011 11495 hybrid_clock.cc:648] HybridClock initialized: now 1786055976065950 us; error 47 us; skew 500 ppm
I20260806 22:39:36.066445 11495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:36 dist-test-slave-h3ft krb5kdc[11337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055976, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:36.070775 11495 init.cc:377] Logged in from keytab as kudu/127.15.198.129@KRBTEST.COM (short username kudu)
I20260806 22:39:36.072984 11495 webserver.cc:460] Webserver started at http://127.15.198.129:44393/ using document root <none> and password file <none>
I20260806 22:39:36.073952 11495 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:36.074122 11495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:36.074519 11495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:36.078406 11495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "ac201fef3ab7422486fd61cb550eab77"
format_stamp: "Formatted at 2026-08-06 22:39:36 on dist-test-slave-h3ft"
server_key: "49ae5f6c6405c27279a7bef165952817"
server_key_iv: "3f24e47938eb8120f68d5249dddf3a38"
server_key_version: "encryptionkey@0"
I20260806 22:39:36.079419 11495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "ac201fef3ab7422486fd61cb550eab77"
format_stamp: "Formatted at 2026-08-06 22:39:36 on dist-test-slave-h3ft"
server_key: "49ae5f6c6405c27279a7bef165952817"
server_key_iv: "3f24e47938eb8120f68d5249dddf3a38"
server_key_version: "encryptionkey@0"
I20260806 22:39:36.086294 11495 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 22:39:36.090940 11516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:36.092913 11495 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:39:36.093179 11495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "ac201fef3ab7422486fd61cb550eab77"
format_stamp: "Formatted at 2026-08-06 22:39:36 on dist-test-slave-h3ft"
server_key: "49ae5f6c6405c27279a7bef165952817"
server_key_iv: "3f24e47938eb8120f68d5249dddf3a38"
server_key_version: "encryptionkey@0"
I20260806 22:39:36.093442 11495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:36.145550 11495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:36.151280 11495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:36.151683 11495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:36.153092 11495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:36.154963 11495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:36.155113 11495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 22:39:36.155292 11495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:36.155373 11495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:36.195914 11495 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:38613
I20260806 22:39:36.195969 11674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:38613 every 8 connection(s)
I20260806 22:39:36.197533 11495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:39:36.204561 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 11495
I20260806 22:39:36.204810 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:39:36.205513 16154 external_mini_cluster.cc:1428] Setting key 638475464e2fe858538d94db4fbf023d
Aug 06 22:39:36 dist-test-slave-h3ft krb5kdc[11337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055976, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.129@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:39:36.223922 11677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:36.200912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:43151 (local address 127.15.198.190:33971)
0806 22:39:36.201484 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:36.201492 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:36.202692 (+ 1200us) server_negotiation.cc:407] Connection header received
0806 22:39:36.204945 (+ 2253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:36.204953 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:36.205125 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:36.205271 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:36.207400 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:36.208527 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:36.211615 (+ 3088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:36.212094 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:36.212215 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:36.217175 (+ 4960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:36.217181 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:36.217184 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:36.217227 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:36.220255 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:36.221120 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:36.221133 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:36.221136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:36.221271 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:36.221774 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:36.221784 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:36.221786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:36.222111 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:36.222289 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:36.223199 (+ 910us) server_negotiation.cc:299] Negotiation successful
0806 22:39:36.223471 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":267,"threads_started":1}
I20260806 22:39:36.225189 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:36.201660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33971 (local address 127.15.198.129:43151)
0806 22:39:36.202488 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:36.202542 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:36.204578 (+ 2036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:36.205516 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:36.205525 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:36.206253 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:36.207208 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:36.207234 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:36.208843 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:36.208850 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:36.211425 (+ 2575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:36.211446 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:36.212387 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:36.212397 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:36.212802 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:36.213938 (+ 1136us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:36.213975 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:36.216854 (+ 2879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:36.220435 (+ 3581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:36.220446 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:36.220469 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:36.220975 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:36.221405 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:36.221410 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:36.221414 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:36.221641 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:36.222255 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:36.222265 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:36.222715 (+ 450us) client_negotiation.cc:769] Sending connection context
0806 22:39:36.223057 (+ 342us) client_negotiation.cc:240] Negotiation successful
0806 22:39:36.223735 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":338,"threads_started":1}
I20260806 22:39:36.227393 11675 heartbeater.cc:347] Connected to a master server at 127.15.198.190:33971
I20260806 22:39:36.227901 11675 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:36.228899 11675 heartbeater.cc:511] Master 127.15.198.190:33971 requested a full tablet report, sending...
I20260806 22:39:36.231866 11412 ts_manager.cc:194] Registered new tserver with Master: ac201fef3ab7422486fd61cb550eab77 (127.15.198.129:38613)
I20260806 22:39:36.233650 11412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.129@KRBTEST.COM'} at 127.15.198.129:43151
WARNING: no policy specified for kudu/127.15.198.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
Entry for principal kudu/127.15.198.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab.
I20260806 22:39:36.296880 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33971
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:33305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:36.729158 11681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:36.729557 11681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:36.729732 11681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:36.738206 11681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:36.738373 11681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:36.738584 11681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:39:36.749890 11681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:33305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.130.keytab
--principal=kudu/127.15.198.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:36.751811 11681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:36.753557 11681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:36.769248 11686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:36.769299 11689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:36.769282 11687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:36.770257 11681 server_base.cc:1034] running on GCE node
I20260806 22:39:36.771119 11681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:36.772295 11681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:36.773557 11681 hybrid_clock.cc:648] HybridClock initialized: now 1786055976773497 us; error 86 us; skew 500 ppm
I20260806 22:39:36.773949 11681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:36 dist-test-slave-h3ft krb5kdc[11337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055976, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:36.777917 11681 init.cc:377] Logged in from keytab as kudu/127.15.198.130@KRBTEST.COM (short username kudu)
I20260806 22:39:36.780058 11681 webserver.cc:460] Webserver started at http://127.15.198.130:38795/ using document root <none> and password file <none>
I20260806 22:39:36.780937 11681 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:36.781111 11681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:36.781504 11681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:36.785125 11681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "dc91a08bb8d945ad863a71d7f27a96cb"
format_stamp: "Formatted at 2026-08-06 22:39:36 on dist-test-slave-h3ft"
server_key: "3d7e820cacebcc56de7e844c68652a3a"
server_key_iv: "6f9538761ebbdc4e33947d63df9b1a55"
server_key_version: "encryptionkey@0"
I20260806 22:39:36.786020 11681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "dc91a08bb8d945ad863a71d7f27a96cb"
format_stamp: "Formatted at 2026-08-06 22:39:36 on dist-test-slave-h3ft"
server_key: "3d7e820cacebcc56de7e844c68652a3a"
server_key_iv: "6f9538761ebbdc4e33947d63df9b1a55"
server_key_version: "encryptionkey@0"
I20260806 22:39:36.793658 11681 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.003s
I20260806 22:39:36.798317 11698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:36.800282 11681 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:39:36.800513 11681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "dc91a08bb8d945ad863a71d7f27a96cb"
format_stamp: "Formatted at 2026-08-06 22:39:36 on dist-test-slave-h3ft"
server_key: "3d7e820cacebcc56de7e844c68652a3a"
server_key_iv: "6f9538761ebbdc4e33947d63df9b1a55"
server_key_version: "encryptionkey@0"
I20260806 22:39:36.800779 11681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:36.821447 11681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:36.826797 11681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:36.827241 11681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:36.828482 11681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:36.830271 11681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:36.830389 11681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:36.830567 11681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:36.830651 11681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:36.877791 11681 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:39891
I20260806 22:39:36.877916 11847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:39891 every 8 connection(s)
I20260806 22:39:36.879760 11681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:39:36.883607 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 11681
I20260806 22:39:36.883860 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:39:36.884529 16154 external_mini_cluster.cc:1428] Setting key 1754a82686c1e67cf454ae66424f0010
Aug 06 22:39:36 dist-test-slave-h3ft krb5kdc[11337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055976, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.130@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:39:36.908725 11850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:36.883513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:53813 (local address 127.15.198.190:33971)
0806 22:39:36.884304 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:36.884311 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:36.885782 (+ 1471us) server_negotiation.cc:407] Connection header received
0806 22:39:36.888353 (+ 2571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:36.888364 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:36.888502 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:36.888657 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:36.890859 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:36.892195 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:36.894198 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:36.895048 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:36.895200 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:36.900529 (+ 5329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:36.900537 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:36.900540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:36.900591 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:36.904250 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:36.905440 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:36.905445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:36.905449 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:36.905632 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:36.906380 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:36.906389 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:36.906394 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:36.906792 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:36.906971 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:36.908134 (+ 1163us) server_negotiation.cc:299] Negotiation successful
0806 22:39:36.908409 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":435,"threads_started":1}
I20260806 22:39:36.910549 11851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:36.884389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33971 (local address 127.15.198.130:53813)
0806 22:39:36.885560 (+ 1171us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:36.885627 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:36.887970 (+ 2343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:36.888946 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:36.888958 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:36.889732 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:36.890628 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:36.890653 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:36.892404 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:36.892409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:36.894008 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:36.894023 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:36.895338 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:36.895343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:36.895607 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:36.896568 (+ 961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:36.896606 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:36.900175 (+ 3569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:36.904433 (+ 4258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:36.904458 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:36.904486 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:36.905253 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:36.905843 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:36.905850 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:36.905856 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:36.906188 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:36.906974 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:36.906986 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:36.907628 (+ 642us) client_negotiation.cc:769] Sending connection context
0806 22:39:36.908015 (+ 387us) client_negotiation.cc:240] Negotiation successful
0806 22:39:36.908731 (+ 716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"thread_start_us":507,"threads_started":1}
I20260806 22:39:36.912830 11848 heartbeater.cc:347] Connected to a master server at 127.15.198.190:33971
I20260806 22:39:36.913241 11848 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:36.914441 11848 heartbeater.cc:511] Master 127.15.198.190:33971 requested a full tablet report, sending...
I20260806 22:39:36.916731 11412 ts_manager.cc:194] Registered new tserver with Master: dc91a08bb8d945ad863a71d7f27a96cb (127.15.198.130:39891)
I20260806 22:39:36.918212 11412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.130@KRBTEST.COM'} at 127.15.198.130:53813
WARNING: no policy specified for kudu/127.15.198.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
Entry for principal kudu/127.15.198.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab.
I20260806 22:39:36.983596 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33971
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:33305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:39:37.237591 11675 heartbeater.cc:503] Master 127.15.198.190:33971 was elected leader, sending a full tablet report...
W20260806 22:39:37.430037 11856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:37.430594 11856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:37.430785 11856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:37.439553 11856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:37.439800 11856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:37.440034 11856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:39:37.451751 11856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:33305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.131.keytab
--principal=kudu/127.15.198.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:33971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:37.453719 11856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:37.455531 11856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:37.472826 11862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:37.473100 11867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:37.472826 11864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:37.474531 11856 server_base.cc:1034] running on GCE node
I20260806 22:39:37.475541 11856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:37.476830 11856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:37.478120 11856 hybrid_clock.cc:648] HybridClock initialized: now 1786055977478070 us; error 49 us; skew 500 ppm
I20260806 22:39:37.478560 11856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:37 dist-test-slave-h3ft krb5kdc[11337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055977, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:37.482777 11856 init.cc:377] Logged in from keytab as kudu/127.15.198.131@KRBTEST.COM (short username kudu)
I20260806 22:39:37.484923 11856 webserver.cc:460] Webserver started at http://127.15.198.131:38937/ using document root <none> and password file <none>
I20260806 22:39:37.485831 11856 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:37.486006 11856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:37.486404 11856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:37.490289 11856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "59d4c0ed488b42ff830a96156551e6c4"
format_stamp: "Formatted at 2026-08-06 22:39:37 on dist-test-slave-h3ft"
server_key: "1aba94f40ce86945cb94697b65d6e4c3"
server_key_iv: "cc2e7ec5e870724462831a5a1d54b69f"
server_key_version: "encryptionkey@0"
I20260806 22:39:37.491406 11856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "59d4c0ed488b42ff830a96156551e6c4"
format_stamp: "Formatted at 2026-08-06 22:39:37 on dist-test-slave-h3ft"
server_key: "1aba94f40ce86945cb94697b65d6e4c3"
server_key_iv: "cc2e7ec5e870724462831a5a1d54b69f"
server_key_version: "encryptionkey@0"
I20260806 22:39:37.498903 11856 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260806 22:39:37.503942 11877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:37.506037 11856 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:39:37.506286 11856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "59d4c0ed488b42ff830a96156551e6c4"
format_stamp: "Formatted at 2026-08-06 22:39:37 on dist-test-slave-h3ft"
server_key: "1aba94f40ce86945cb94697b65d6e4c3"
server_key_iv: "cc2e7ec5e870724462831a5a1d54b69f"
server_key_version: "encryptionkey@0"
I20260806 22:39:37.506567 11856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:37.526238 11856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:37.532209 11856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:37.532632 11856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:37.533983 11856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:37.535884 11856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:37.536008 11856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:37.536192 11856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:37.536273 11856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:37.577569 11856 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:39051
I20260806 22:39:37.577613 12031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:39051 every 8 connection(s)
I20260806 22:39:37.579272 11856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:39:37.579952 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 11856
I20260806 22:39:37.580226 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:39:37.580955 16154 external_mini_cluster.cc:1428] Setting key 3090bede26c2436fe1be43514ffccee9
Aug 06 22:39:37 dist-test-slave-h3ft krb5kdc[11337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055977, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.131@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:39:37.606153 12036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:37.583111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:41489 (local address 127.15.198.190:33971)
0806 22:39:37.583693 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:37.583698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:37.585280 (+ 1582us) server_negotiation.cc:407] Connection header received
0806 22:39:37.587737 (+ 2457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:37.587742 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:37.587838 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:37.587951 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:37.590003 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.590898 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.592863 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.593383 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.593495 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:37.598506 (+ 5011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:37.598512 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:37.598516 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:37.598572 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:37.601873 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:37.602843 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:37.602848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:37.602850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:37.603183 (+ 333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:37.603751 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:37.603757 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:37.603761 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:37.604124 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:37.604318 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:37.605337 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0806 22:39:37.605636 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":295,"threads_started":1}
I20260806 22:39:37.607548 12037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:37.583992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33971 (local address 127.15.198.131:41489)
0806 22:39:37.585069 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:37.585134 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:37.587403 (+ 2269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:37.588247 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:37.588258 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:37.589026 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:37.589796 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:37.589820 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.591181 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.591186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:37.592713 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:37.592727 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.593636 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.593642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:37.593932 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:37.594807 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:37.594835 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:37.598259 (+ 3424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:37.602073 (+ 3814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:37.602085 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:37.602113 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:37.602688 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:37.603317 (+ 629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:37.603323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:37.603327 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:37.603605 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:37.604277 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:37.604289 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:37.604795 (+ 506us) client_negotiation.cc:769] Sending connection context
0806 22:39:37.605221 (+ 426us) client_negotiation.cc:240] Negotiation successful
0806 22:39:37.605943 (+ 722us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"thread_start_us":454,"threads_started":1}
I20260806 22:39:37.609748 12034 heartbeater.cc:347] Connected to a master server at 127.15.198.190:33971
I20260806 22:39:37.610200 12034 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:37.611477 12034 heartbeater.cc:511] Master 127.15.198.190:33971 requested a full tablet report, sending...
I20260806 22:39:37.613315 11412 ts_manager.cc:194] Registered new tserver with Master: 59d4c0ed488b42ff830a96156551e6c4 (127.15.198.131:39051)
I20260806 22:39:37.614427 11412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.198.131@KRBTEST.COM'} at 127.15.198.131:41489
I20260806 22:39:37.624173 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:39:37.644456 12036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:37.629368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34624 (local address 127.15.198.190:33971)
0806 22:39:37.629602 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:37.629607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:37.630189 (+ 582us) server_negotiation.cc:407] Connection header received
0806 22:39:37.630433 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:37.630438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:37.630532 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:37.630638 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:37.632552 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.633409 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.634936 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.635433 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.635554 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:37.638464 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:37.638470 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:37.638474 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:37.638519 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:37.641630 (+ 3111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:37.642369 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:37.642373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:37.642375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:37.642521 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:37.642873 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:37.642876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:37.642878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:37.643346 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:37.643575 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:37.643954 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 22:39:37.644179 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 22:39:37.649865 11412 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:39:37.658082 11412 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:39:37.699638 12050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:37.673947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:38613 (local address 127.0.0.1:54642)
0806 22:39:37.681625 (+ 7678us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:37.681704 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:37.682008 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:37.690275 (+ 8267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:37.690284 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:37.690315 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:37.690939 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:37.690959 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.693539 (+ 2580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.693544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:37.696313 (+ 2769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:37.696330 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.698130 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.698136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:37.698407 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:37.698501 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 22:39:37.698658 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 22:39:37.699087 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7282,"thread_start_us":7056,"threads_started":1}
I20260806 22:39:37.700820 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:37.674101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54642 (local address 127.15.198.129:38613)
0806 22:39:37.688738 (+ 14637us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:37.688746 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:37.688782 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:39:37.688985 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:37.689000 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:37.689455 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:37.691360 (+ 1905us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:37.691709 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.693372 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.696508 (+ 3136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.697915 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.698150 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:37.700044 (+ 1894us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:37.700155 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 22:39:37.700383 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14430,"thread_start_us":14187,"threads_started":1}
I20260806 22:39:37.701217 12054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:37.682866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41524 (local address 127.15.198.130:39891)
0806 22:39:37.691883 (+ 9017us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:37.691890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:37.691920 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:39:37.692029 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:37.692036 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:37.692243 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:37.692490 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:37.693097 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.694932 (+ 1835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.697730 (+ 2798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.699600 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.700364 (+ 764us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:37.700548 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:37.700637 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 22:39:37.700825 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8833,"thread_start_us":8670,"threads_started":1}
I20260806 22:39:37.701660 12055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:37.688745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:39891 (local address 127.0.0.1:41524)
0806 22:39:37.689424 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:37.689445 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:37.689601 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:37.692479 (+ 2878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:37.692484 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:37.692514 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:37.692914 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:37.692923 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.695185 (+ 2262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.695191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:37.697503 (+ 2312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:37.697518 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.700032 (+ 2514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.700039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:37.700247 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:37.700283 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:39:37.700366 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:39:37.700515 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":6,"thread_start_us":624,"threads_started":1}
I20260806 22:39:37.702286 12052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:37.674643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:39051 (local address 127.0.0.1:51296)
0806 22:39:37.688669 (+ 14026us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:37.688693 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:37.688932 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:37.690336 (+ 1404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:37.690344 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:37.690386 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:37.691151 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:37.691170 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.696533 (+ 5363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.696538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:37.698727 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:37.698744 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.701430 (+ 2686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.701437 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:37.701616 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:37.701653 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 22:39:37.701727 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:39:37.701860 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13844,"mutex_wait_us":5,"spinlock_wait_cycles":4480,"thread_start_us":15035,"threads_started":1}
I20260806 22:39:37.703413 12053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:37.675228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51296 (local address 127.15.198.131:39051)
0806 22:39:37.687658 (+ 12430us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:37.687676 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:37.688884 (+ 1208us) server_negotiation.cc:407] Connection header received
0806 22:39:37.689294 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:37.689308 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:37.689573 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:37.689820 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:37.691423 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.693383 (+ 1960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.698950 (+ 5567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.701199 (+ 2249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.701432 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:37.701846 (+ 414us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:37.701964 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 22:39:37.702166 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12085,"thread_start_us":12457,"threads_started":1}
I20260806 22:39:37.706226 11588 tablet_service.cc:1467] Processing CreateTablet for tablet d11fe15903a74d44b451da0acd703157 (DEFAULT_TABLE table=test-table [id=b16f1a1bc1f94b1a8a7fc8cc56b5416c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:37.706221 11760 tablet_service.cc:1467] Processing CreateTablet for tablet d11fe15903a74d44b451da0acd703157 (DEFAULT_TABLE table=test-table [id=b16f1a1bc1f94b1a8a7fc8cc56b5416c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:37.707945 11588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d11fe15903a74d44b451da0acd703157. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:37.707975 11760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d11fe15903a74d44b451da0acd703157. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:37.707562 11952 tablet_service.cc:1467] Processing CreateTablet for tablet d11fe15903a74d44b451da0acd703157 (DEFAULT_TABLE table=test-table [id=b16f1a1bc1f94b1a8a7fc8cc56b5416c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:37.709069 11952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d11fe15903a74d44b451da0acd703157. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:37.722790 12056 tablet_bootstrap.cc:492] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77: Bootstrap starting.
I20260806 22:39:37.725225 12057 tablet_bootstrap.cc:492] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb: Bootstrap starting.
I20260806 22:39:37.727938 12058 tablet_bootstrap.cc:492] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4: Bootstrap starting.
I20260806 22:39:37.729313 12056 tablet_bootstrap.cc:654] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:37.729425 12057 tablet_bootstrap.cc:654] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:37.731103 12057 log.cc:824] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:37.731138 12056 log.cc:824] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:37.731624 12058 tablet_bootstrap.cc:654] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:37.732872 12058 log.cc:824] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:37.736562 12058 tablet_bootstrap.cc:492] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4: No bootstrap required, opened a new log
I20260806 22:39:37.736649 12056 tablet_bootstrap.cc:492] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77: No bootstrap required, opened a new log
I20260806 22:39:37.736652 12057 tablet_bootstrap.cc:492] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb: No bootstrap required, opened a new log
I20260806 22:39:37.736961 12058 ts_tablet_manager.cc:1397] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 22:39:37.736975 12057 ts_tablet_manager.cc:1397] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260806 22:39:37.736971 12056 ts_tablet_manager.cc:1397] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260806 22:39:37.745968 12058 raft_consensus.cc:348] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac201fef3ab7422486fd61cb550eab77" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38613 } } peers { permanent_uuid: "dc91a08bb8d945ad863a71d7f27a96cb" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39891 } } peers { permanent_uuid: "59d4c0ed488b42ff830a96156551e6c4" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 39051 } }
I20260806 22:39:37.746572 12058 raft_consensus.cc:374] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:37.746716 12058 raft_consensus.cc:729] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59d4c0ed488b42ff830a96156551e6c4, State: Initialized, Role: FOLLOWER
I20260806 22:39:37.747570 12058 consensus_queue.cc:260] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac201fef3ab7422486fd61cb550eab77" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38613 } } peers { permanent_uuid: "dc91a08bb8d945ad863a71d7f27a96cb" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39891 } } peers { permanent_uuid: "59d4c0ed488b42ff830a96156551e6c4" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 39051 } }
I20260806 22:39:37.748989 12034 heartbeater.cc:503] Master 127.15.198.190:33971 was elected leader, sending a full tablet report...
I20260806 22:39:37.749091 12056 raft_consensus.cc:348] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac201fef3ab7422486fd61cb550eab77" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38613 } } peers { permanent_uuid: "dc91a08bb8d945ad863a71d7f27a96cb" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39891 } } peers { permanent_uuid: "59d4c0ed488b42ff830a96156551e6c4" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 39051 } }
I20260806 22:39:37.749086 12057 raft_consensus.cc:348] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac201fef3ab7422486fd61cb550eab77" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38613 } } peers { permanent_uuid: "dc91a08bb8d945ad863a71d7f27a96cb" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39891 } } peers { permanent_uuid: "59d4c0ed488b42ff830a96156551e6c4" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 39051 } }
I20260806 22:39:37.749625 12056 raft_consensus.cc:374] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:37.749624 12057 raft_consensus.cc:374] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:37.749742 12057 raft_consensus.cc:729] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc91a08bb8d945ad863a71d7f27a96cb, State: Initialized, Role: FOLLOWER
I20260806 22:39:37.749734 12056 raft_consensus.cc:729] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac201fef3ab7422486fd61cb550eab77, State: Initialized, Role: FOLLOWER
I20260806 22:39:37.749857 12058 ts_tablet_manager.cc:1428] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260806 22:39:37.750573 12056 consensus_queue.cc:260] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac201fef3ab7422486fd61cb550eab77" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38613 } } peers { permanent_uuid: "dc91a08bb8d945ad863a71d7f27a96cb" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39891 } } peers { permanent_uuid: "59d4c0ed488b42ff830a96156551e6c4" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 39051 } }
I20260806 22:39:37.750588 12057 consensus_queue.cc:260] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac201fef3ab7422486fd61cb550eab77" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38613 } } peers { permanent_uuid: "dc91a08bb8d945ad863a71d7f27a96cb" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39891 } } peers { permanent_uuid: "59d4c0ed488b42ff830a96156551e6c4" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 39051 } }
I20260806 22:39:37.752928 12056 ts_tablet_manager.cc:1428] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260806 22:39:37.753168 12057 ts_tablet_manager.cc:1428] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
I20260806 22:39:37.754549 11848 heartbeater.cc:503] Master 127.15.198.190:33971 was elected leader, sending a full tablet report...
I20260806 22:39:37.769042 12063 raft_consensus.cc:482] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:39:37.769519 12063 raft_consensus.cc:504] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac201fef3ab7422486fd61cb550eab77" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38613 } } peers { permanent_uuid: "dc91a08bb8d945ad863a71d7f27a96cb" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39891 } } peers { permanent_uuid: "59d4c0ed488b42ff830a96156551e6c4" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 39051 } }
I20260806 22:39:37.772018 12063 leader_election.cc:288] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac201fef3ab7422486fd61cb550eab77 (127.15.198.129:38613), dc91a08bb8d945ad863a71d7f27a96cb (127.15.198.130:39891)
I20260806 22:39:37.778844 12037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:37.772348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:38613 (local address 127.15.198.131:58947)
0806 22:39:37.772614 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:37.772641 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:37.772790 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:37.773553 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:37.773559 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:37.773600 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:37.773865 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:37.773877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.775131 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.775137 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:37.776890 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:37.776908 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.778101 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.778107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:37.778342 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:37.778389 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 22:39:37.778479 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 22:39:37.778589 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":1}
I20260806 22:39:37.779021 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:37.772718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:58947 (local address 127.15.198.129:38613)
0806 22:39:37.772977 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:37.772984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:37.773007 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:39:37.773114 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:37.773120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:37.773222 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:37.773326 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:37.774061 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.774919 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.777079 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.777938 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.778074 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:37.778141 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:37.778550 (+ 409us) server_negotiation.cc:299] Negotiation successful
0806 22:39:37.778718 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 22:39:37.780727 12054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:37.773285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:38513 (local address 127.15.198.130:39891)
0806 22:39:37.773584 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:37.773594 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:37.774361 (+ 767us) server_negotiation.cc:407] Connection header received
0806 22:39:37.774585 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:37.774590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:37.774703 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:37.774817 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:37.775635 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.776386 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.778411 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.779762 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.779893 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:37.779952 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:37.780386 (+ 434us) server_negotiation.cc:299] Negotiation successful
0806 22:39:37.780496 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 22:39:37.780423 11614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d11fe15903a74d44b451da0acd703157" candidate_uuid: "59d4c0ed488b42ff830a96156551e6c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac201fef3ab7422486fd61cb550eab77" is_pre_election: true
I20260806 22:39:37.780993 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:37.772805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:39891 (local address 127.15.198.131:38513)
0806 22:39:37.774226 (+ 1421us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:37.774247 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:37.774408 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:37.774833 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:37.774838 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:37.774865 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:37.775419 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:37.775432 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.776550 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.776555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:37.778241 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:37.778256 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.780000 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.780007 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:37.780192 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:37.780245 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 22:39:37.780362 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 22:39:37.780517 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1320,"thread_start_us":1052,"threads_started":1}
I20260806 22:39:37.781293 11614 raft_consensus.cc:2454] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59d4c0ed488b42ff830a96156551e6c4 in term 0.
I20260806 22:39:37.782191 11883 leader_election.cc:302] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59d4c0ed488b42ff830a96156551e6c4, ac201fef3ab7422486fd61cb550eab77; no voters:
I20260806 22:39:37.782275 11793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d11fe15903a74d44b451da0acd703157" candidate_uuid: "59d4c0ed488b42ff830a96156551e6c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc91a08bb8d945ad863a71d7f27a96cb" is_pre_election: true
I20260806 22:39:37.782687 12063 raft_consensus.cc:2781] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:39:37.782850 12063 raft_consensus.cc:482] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:39:37.782986 12063 raft_consensus.cc:3037] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:37.782938 11793 raft_consensus.cc:2454] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59d4c0ed488b42ff830a96156551e6c4 in term 0.
I20260806 22:39:37.785857 12063 raft_consensus.cc:504] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac201fef3ab7422486fd61cb550eab77" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38613 } } peers { permanent_uuid: "dc91a08bb8d945ad863a71d7f27a96cb" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39891 } } peers { permanent_uuid: "59d4c0ed488b42ff830a96156551e6c4" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 39051 } }
I20260806 22:39:37.786646 12063 leader_election.cc:288] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [CANDIDATE]: Term 1 election: Requested vote from peers ac201fef3ab7422486fd61cb550eab77 (127.15.198.129:38613), dc91a08bb8d945ad863a71d7f27a96cb (127.15.198.130:39891)
I20260806 22:39:37.787415 11614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d11fe15903a74d44b451da0acd703157" candidate_uuid: "59d4c0ed488b42ff830a96156551e6c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac201fef3ab7422486fd61cb550eab77"
I20260806 22:39:37.787403 11793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d11fe15903a74d44b451da0acd703157" candidate_uuid: "59d4c0ed488b42ff830a96156551e6c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc91a08bb8d945ad863a71d7f27a96cb"
I20260806 22:39:37.787696 11793 raft_consensus.cc:3037] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:37.787703 11614 raft_consensus.cc:3037] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:37.790402 11614 raft_consensus.cc:2454] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59d4c0ed488b42ff830a96156551e6c4 in term 1.
I20260806 22:39:37.790402 11793 raft_consensus.cc:2454] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59d4c0ed488b42ff830a96156551e6c4 in term 1.
I20260806 22:39:37.791079 11883 leader_election.cc:302] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59d4c0ed488b42ff830a96156551e6c4, ac201fef3ab7422486fd61cb550eab77; no voters:
I20260806 22:39:37.791482 12063 raft_consensus.cc:2781] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:37.792078 12063 raft_consensus.cc:686] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [term 1 LEADER]: Becoming Leader. State: Replica: 59d4c0ed488b42ff830a96156551e6c4, State: Running, Role: LEADER
I20260806 22:39:37.792865 12063 consensus_queue.cc:237] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac201fef3ab7422486fd61cb550eab77" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38613 } } peers { permanent_uuid: "dc91a08bb8d945ad863a71d7f27a96cb" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39891 } } peers { permanent_uuid: "59d4c0ed488b42ff830a96156551e6c4" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 39051 } }
I20260806 22:39:37.800083 11408 catalog_manager.cc:5495] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59d4c0ed488b42ff830a96156551e6c4 (127.15.198.131). New cstate: current_term: 1 leader_uuid: "59d4c0ed488b42ff830a96156551e6c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac201fef3ab7422486fd61cb550eab77" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc91a08bb8d945ad863a71d7f27a96cb" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59d4c0ed488b42ff830a96156551e6c4" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 39051 } health_report { overall_health: HEALTHY } } }
W20260806 22:39:37.831717 12035 tablet.cc:2369] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:39:37.833837 12053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:37.827156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51308 (local address 127.15.198.131:39051)
0806 22:39:37.827437 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:37.827446 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:37.827514 (+ 68us) server_negotiation.cc:407] Connection header received
0806 22:39:37.827846 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:37.827852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:37.827968 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:37.828173 (+ 205us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:39:37.828632 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.829458 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.830995 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.831537 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.831723 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:37.832430 (+ 707us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:39:37.833129 (+ 699us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:39:37.833334 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:37.833389 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 22:39:37.833510 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 22:39:37.846037 11793 raft_consensus.cc:1261] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb [term 1 FOLLOWER]: Refusing update from remote peer 59d4c0ed488b42ff830a96156551e6c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:37.845999 11614 raft_consensus.cc:1261] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77 [term 1 FOLLOWER]: Refusing update from remote peer 59d4c0ed488b42ff830a96156551e6c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:37.847497 12063 consensus_queue.cc:1035] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac201fef3ab7422486fd61cb550eab77" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:37.848008 12070 consensus_queue.cc:1035] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc91a08bb8d945ad863a71d7f27a96cb" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 39891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:37.863571 12077 mvcc.cc:205] Tried to move back new op lower bound from 7315685285240778752 to 7315685285045972992. Current Snapshot: MvccSnapshot[applied={T|T < 7315685285240778752}]
I20260806 22:39:37.883123 11411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:39:37.883572 11411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:39:37.889220 11411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ab4083310d84446ae9b6452a6253f1c: Sending DeleteTablet for 3 replicas of tablet d11fe15903a74d44b451da0acd703157
I20260806 22:39:37.890916 11952 tablet_service.cc:1514] Processing DeleteTablet for tablet d11fe15903a74d44b451da0acd703157 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:37 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:51296
I20260806 22:39:37.891215 11760 tablet_service.cc:1514] Processing DeleteTablet for tablet d11fe15903a74d44b451da0acd703157 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:37 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:41524
I20260806 22:39:37.891921 11588 tablet_service.cc:1514] Processing DeleteTablet for tablet d11fe15903a74d44b451da0acd703157 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:37 UTC) from {username='kudu', principal='kudu/127.15.198.190@KRBTEST.COM'} at 127.0.0.1:54642
I20260806 22:39:37.899391 12088 tablet_replica.cc:330] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb: stopping tablet replica
I20260806 22:39:37.899827 12089 tablet_replica.cc:330] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77: stopping tablet replica
I20260806 22:39:37.900014 12088 raft_consensus.cc:2229] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:39:37.900527 12089 raft_consensus.cc:2229] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:39:37.901155 12089 raft_consensus.cc:2258] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:37.904336 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 11495
I20260806 22:39:37.904323 12089 ts_tablet_manager.cc:1905] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:37.907791 12087 tablet_replica.cc:330] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4: stopping tablet replica
I20260806 22:39:37.908422 12087 raft_consensus.cc:2229] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:39:37.909379 12087 raft_consensus.cc:2258] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:37.912627 12087 ts_tablet_manager.cc:1905] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:37.915437 12089 ts_tablet_manager.cc:1918] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:37.915704 12089 log.cc:1196] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/d11fe15903a74d44b451da0acd703157
I20260806 22:39:37.916409 12089 ts_tablet_manager.cc:1939] T d11fe15903a74d44b451da0acd703157 P ac201fef3ab7422486fd61cb550eab77: Deleting consensus metadata
I20260806 22:39:37.919018 11386 catalog_manager.cc:4841] TS ac201fef3ab7422486fd61cb550eab77 (127.15.198.129:38613): tablet d11fe15903a74d44b451da0acd703157 (table test-table [id=b16f1a1bc1f94b1a8a7fc8cc56b5416c]) successfully deleted
I20260806 22:39:37.920053 12088 raft_consensus.cc:2258] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:37.921438 12087 ts_tablet_manager.cc:1918] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:37.921646 12087 log.cc:1196] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/d11fe15903a74d44b451da0acd703157
I20260806 22:39:37.922256 12087 ts_tablet_manager.cc:1939] T d11fe15903a74d44b451da0acd703157 P 59d4c0ed488b42ff830a96156551e6c4: Deleting consensus metadata
I20260806 22:39:37.923863 12088 ts_tablet_manager.cc:1905] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:37.924535 11384 catalog_manager.cc:4841] TS 59d4c0ed488b42ff830a96156551e6c4 (127.15.198.131:39051): tablet d11fe15903a74d44b451da0acd703157 (table test-table [id=b16f1a1bc1f94b1a8a7fc8cc56b5416c]) successfully deleted
I20260806 22:39:37.930886 12088 ts_tablet_manager.cc:1918] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:37.931259 12088 log.cc:1196] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/d11fe15903a74d44b451da0acd703157
I20260806 22:39:37.932024 12088 ts_tablet_manager.cc:1939] T d11fe15903a74d44b451da0acd703157 P dc91a08bb8d945ad863a71d7f27a96cb: Deleting consensus metadata
Aug 06 22:39:37 dist-test-slave-h3ft krb5kdc[11337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.129@KRBTEST.COM
I20260806 22:39:37.934672 11386 catalog_manager.cc:4841] TS dc91a08bb8d945ad863a71d7f27a96cb (127.15.198.130:39891): tablet d11fe15903a74d44b451da0acd703157 (table test-table [id=b16f1a1bc1f94b1a8a7fc8cc56b5416c]) successfully deleted
I20260806 22:39:37.943447 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:37.906357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54654 (local address 127.15.198.129:38613)
0806 22:39:37.906589 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:37.906598 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:37.922410 (+ 15812us) server_negotiation.cc:407] Connection header received
0806 22:39:37.923057 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:37.923066 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:37.923177 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:37.923274 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:37.925677 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.926921 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.928990 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:37.929683 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:37.929834 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:37.936389 (+ 6555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:37.936411 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:37.936432 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:37.936484 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:37.940062 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:37.941081 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:37.941089 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:37.941092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:37.941241 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:37.941687 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:37.941690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:37.941693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:37.942128 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:37.942402 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:37.942800 (+ 398us) server_negotiation.cc:299] Negotiation successful
0806 22:39:37.943052 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 22:39:37.944373 11669 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:38.054379 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 11495
I20260806 22:39:38.101469 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 11681
Aug 06 22:39:38 dist-test-slave-h3ft krb5kdc[11337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.130@KRBTEST.COM
I20260806 22:39:38.122766 12054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:38.103168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41536 (local address 127.15.198.130:39891)
0806 22:39:38.103465 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:38.103472 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:38.103495 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:39:38.103587 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:38.103593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:38.103691 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:38.103828 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:38.105511 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:38.106844 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:38.108404 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:38.109052 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:38.109195 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:38.115697 (+ 6502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:38.115762 (+ 65us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:38.115793 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:38.115861 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:38.119472 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:38.120341 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:38.120356 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:38.120359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:38.120500 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:38.120933 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:38.120938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:38.120940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:38.121401 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:38.121764 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:38.122096 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 22:39:38.122364 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260806 22:39:38.123725 11843 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:38.225744 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 11681
I20260806 22:39:38.255301 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 11856
Aug 06 22:39:38 dist-test-slave-h3ft krb5kdc[11337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.131@KRBTEST.COM
I20260806 22:39:38.274394 12053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:38.256397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51320 (local address 127.15.198.131:39051)
0806 22:39:38.256649 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:38.256656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:38.256684 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:39:38.256961 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:38.256967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:38.257062 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:38.257192 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:38.258831 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:38.259737 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:38.261305 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:38.261790 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:38.262010 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:38.267328 (+ 5318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:38.267355 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:38.267384 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:38.267458 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:38.271178 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:38.271969 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:38.271978 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:38.271981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:38.272140 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:38.272548 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:38.272554 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:38.272558 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:38.273016 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:38.273425 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:38.273675 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 22:39:38.273946 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260806 22:39:38.275296 12027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:38.419054 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 11856
I20260806 22:39:38.453084 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 11356
I20260806 22:39:38.454573 11452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:38.585343 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 11356
2026-08-06T22:39:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5620 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T22:39:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:39:38Z Disabled control of system clock
I20260806 22:39:38.720566 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:46555
--webserver_interface=127.15.198.190
--webserver_port=0
--builtin_ntp_servers=127.15.198.148:36509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:46555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:39:39.172130 12104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:39.172616 12104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:39.172721 12104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:39.181133 12104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:39.181303 12104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:39.181346 12104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:39.181414 12104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:39.181464 12104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:39.192507 12104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:46555
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:46555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:39.194355 12104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:39.196784 12104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:39.211360 12112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:39.211406 12116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:39.211380 12113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:39.212442 12104 server_base.cc:1034] running on GCE node
I20260806 22:39:39.213555 12104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:39.215559 12104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:39.216830 12104 hybrid_clock.cc:648] HybridClock initialized: now 1786055979216782 us; error 43 us; skew 500 ppm
I20260806 22:39:39.217269 12104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:39:39.220192 12104 webserver.cc:460] Webserver started at http://127.15.198.190:45313/ using document root <none> and password file <none>
I20260806 22:39:39.221109 12104 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:39.221259 12104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:39.221673 12104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:39.225418 12104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "7b0063735de34357963d9fbf96ec110a"
format_stamp: "Formatted at 2026-08-06 22:39:39 on dist-test-slave-h3ft"
server_key: "fd54a58e972bf380b7ccb52091eedead"
server_key_iv: "33b9a75b4ee99f5510a1752badb564c7"
server_key_version: "encryptionkey@0"
I20260806 22:39:39.226405 12104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "7b0063735de34357963d9fbf96ec110a"
format_stamp: "Formatted at 2026-08-06 22:39:39 on dist-test-slave-h3ft"
server_key: "fd54a58e972bf380b7ccb52091eedead"
server_key_iv: "33b9a75b4ee99f5510a1752badb564c7"
server_key_version: "encryptionkey@0"
I20260806 22:39:39.232973 12104 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.003s
I20260806 22:39:39.237587 12123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:39.239344 12104 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:39:39.239584 12104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "7b0063735de34357963d9fbf96ec110a"
format_stamp: "Formatted at 2026-08-06 22:39:39 on dist-test-slave-h3ft"
server_key: "fd54a58e972bf380b7ccb52091eedead"
server_key_iv: "33b9a75b4ee99f5510a1752badb564c7"
server_key_version: "encryptionkey@0"
I20260806 22:39:39.239872 12104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:39.257652 12104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:39.258831 12104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:39.259268 12104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:39.281563 12104 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:46555
I20260806 22:39:39.281591 12195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:46555 every 8 connection(s)
I20260806 22:39:39.283494 12104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:39:39.286772 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 12104
I20260806 22:39:39.287082 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:39:39.287631 16154 external_mini_cluster.cc:1428] Setting key d77e8fa4bd01d9aa9de69f0abbc4f487
I20260806 22:39:39.288976 12196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:39.303635 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:39.290035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49532 (local address 127.15.198.190:46555)
0806 22:39:39.291324 (+ 1289us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:39.291339 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:39.291392 (+ 53us) server_negotiation.cc:407] Connection header received
0806 22:39:39.293249 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:39.293277 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:39.293858 (+ 581us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:39.294343 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:39.295073 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:39.296549 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:39.298148 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:39.298883 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:39.299118 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:39.300623 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:39.300632 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:39.300650 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:39.301380 (+ 730us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:39.301491 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:39.301528 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:39.301691 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:39.301945 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 22:39:39.302353 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":453,"threads_started":1}
I20260806 22:39:39.304169 12196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a: Bootstrap starting.
I20260806 22:39:39.308350 12196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:39.309549 12196 log.cc:824] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:39.313159 12196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a: No bootstrap required, opened a new log
I20260806 22:39:39.321501 12196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b0063735de34357963d9fbf96ec110a" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46555 } }
I20260806 22:39:39.321934 12196 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:39.322028 12196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b0063735de34357963d9fbf96ec110a, State: Initialized, Role: FOLLOWER
I20260806 22:39:39.322707 12196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b0063735de34357963d9fbf96ec110a" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46555 } }
I20260806 22:39:39.322950 12196 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:39:39.323120 12196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:39:39.323282 12196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:39.325716 12196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b0063735de34357963d9fbf96ec110a" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46555 } }
I20260806 22:39:39.326543 12196 leader_election.cc:302] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b0063735de34357963d9fbf96ec110a; no voters:
I20260806 22:39:39.327307 12196 leader_election.cc:288] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:39:39.327486 12202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:39.328330 12202 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [term 1 LEADER]: Becoming Leader. State: Replica: 7b0063735de34357963d9fbf96ec110a, State: Running, Role: LEADER
I20260806 22:39:39.328951 12202 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b0063735de34357963d9fbf96ec110a" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46555 } }
I20260806 22:39:39.329855 12196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:39.332540 12203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b0063735de34357963d9fbf96ec110a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b0063735de34357963d9fbf96ec110a" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46555 } } }
I20260806 22:39:39.333113 12203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:39.333170 12204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b0063735de34357963d9fbf96ec110a. Latest consensus state: current_term: 1 leader_uuid: "7b0063735de34357963d9fbf96ec110a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b0063735de34357963d9fbf96ec110a" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46555 } } }
I20260806 22:39:39.333609 12204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:39.340925 12213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:39:39.347743 12213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:39:39.359826 12213 catalog_manager.cc:1347] Generated new cluster ID: 253a63ddc59547aabf524182a002c747
I20260806 22:39:39.360021 12213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:39:39.369494 12213 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:39:39.371227 12213 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:39:39.384531 12213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a: Generated new TSK 0
I20260806 22:39:39.385816 12213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:39:39.395823 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.129:0
--local_ip_for_outbound_sockets=127.15.198.129
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:46555
--builtin_ntp_servers=127.15.198.148:36509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:39:39.861394 12225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:39.861835 12225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:39.861964 12225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:39.871333 12225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:39.871495 12225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:39.871634 12225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.129
I20260806 22:39:39.883560 12225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.198.129
--webserver_port=0
--tserver_master_addrs=127.15.198.190:46555
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.129
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:39.885666 12225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:39.888825 12225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:39.907300 12235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:39.907382 12231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:39.907225 12232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:39.908601 12225 server_base.cc:1034] running on GCE node
I20260806 22:39:39.909498 12225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:39.910904 12225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:39.912199 12225 hybrid_clock.cc:648] HybridClock initialized: now 1786055979912114 us; error 92 us; skew 500 ppm
I20260806 22:39:39.912663 12225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:39:39.916244 12225 webserver.cc:460] Webserver started at http://127.15.198.129:46317/ using document root <none> and password file <none>
I20260806 22:39:39.917351 12225 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:39.917539 12225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:39.918010 12225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:39.922338 12225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/data/instance:
uuid: "60336ea1ee054eec84058b6c0bdccf4e"
format_stamp: "Formatted at 2026-08-06 22:39:39 on dist-test-slave-h3ft"
server_key: "e2523b5df655a545e756ce05c625c579"
server_key_iv: "00022b36e42e4ca19e15e52a6753f12a"
server_key_version: "encryptionkey@0"
I20260806 22:39:39.923655 12225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance:
uuid: "60336ea1ee054eec84058b6c0bdccf4e"
format_stamp: "Formatted at 2026-08-06 22:39:39 on dist-test-slave-h3ft"
server_key: "e2523b5df655a545e756ce05c625c579"
server_key_iv: "00022b36e42e4ca19e15e52a6753f12a"
server_key_version: "encryptionkey@0"
I20260806 22:39:39.931707 12225 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260806 22:39:39.937350 12245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:39.939728 12225 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.003s
I20260806 22:39:39.940053 12225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/wal
uuid: "60336ea1ee054eec84058b6c0bdccf4e"
format_stamp: "Formatted at 2026-08-06 22:39:39 on dist-test-slave-h3ft"
server_key: "e2523b5df655a545e756ce05c625c579"
server_key_iv: "00022b36e42e4ca19e15e52a6753f12a"
server_key_version: "encryptionkey@0"
I20260806 22:39:39.940423 12225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:39.968123 12225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:39.969367 12225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:39.969799 12225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:39.971159 12225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:39.973227 12225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:39.973376 12225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:39.973526 12225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:39.973624 12225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:40.017547 12225 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.129:33845
I20260806 22:39:40.017611 12395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.129:33845 every 8 connection(s)
I20260806 22:39:40.019572 12225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/data/info.pb
I20260806 22:39:40.021898 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 12225
I20260806 22:39:40.022141 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/wal/instance
I20260806 22:39:40.022819 16154 external_mini_cluster.cc:1428] Setting key c8781177dc7f8f6fcd7ce42fec0fef53
I20260806 22:39:40.026865 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.130:0
--local_ip_for_outbound_sockets=127.15.198.130
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:46555
--builtin_ntp_servers=127.15.198.148:36509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:39:40.037928 12400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:40.023813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.129:48871 (local address 127.15.198.190:46555)
0806 22:39:40.024496 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:40.024503 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:40.026315 (+ 1812us) server_negotiation.cc:407] Connection header received
0806 22:39:40.028706 (+ 2391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:40.028712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:40.028833 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:40.028977 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:40.031273 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:40.032174 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:40.034168 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:40.034684 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:40.034855 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:40.036813 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:40.036819 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:40.036823 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:40.036852 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:40.036905 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:40.036919 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:40.037030 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:40.037339 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 22:39:40.037484 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":341,"threads_started":1}
I20260806 22:39:40.039455 12401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:40.024730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:46555 (local address 127.15.198.129:48871)
0806 22:39:40.026107 (+ 1377us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:40.026165 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:40.028349 (+ 2184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:40.029307 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:40.029318 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:40.029450 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:40.030960 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:40.030994 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:40.032397 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:40.032410 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:40.033960 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:40.033978 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:40.034989 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:40.034998 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:40.035352 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:40.036318 (+ 966us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:39:40.036321 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:40.036379 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:39:40.036503 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:40.037030 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:40.037046 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:40.037072 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 22:39:40.037299 (+ 227us) client_negotiation.cc:240] Negotiation successful
0806 22:39:40.037872 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":449,"threads_started":1}
I20260806 22:39:40.041759 12396 heartbeater.cc:347] Connected to a master server at 127.15.198.190:46555
I20260806 22:39:40.042302 12396 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:40.043444 12396 heartbeater.cc:511] Master 127.15.198.190:46555 requested a full tablet report, sending...
I20260806 22:39:40.046993 12144 ts_manager.cc:194] Registered new tserver with Master: 60336ea1ee054eec84058b6c0bdccf4e (127.15.198.129:33845)
I20260806 22:39:40.049235 12144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.198.129:48871
W20260806 22:39:40.472466 12402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:40.472958 12402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:40.473129 12402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:40.481519 12402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:40.481673 12402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:40.481900 12402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.130
I20260806 22:39:40.492305 12402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.198.130
--webserver_port=0
--tserver_master_addrs=127.15.198.190:46555
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.130
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:40.494210 12402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:40.496722 12402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:40.513540 12409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:40.513633 12410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:40.513698 12413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:40.515120 12402 server_base.cc:1034] running on GCE node
I20260806 22:39:40.515998 12402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:40.517302 12402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:40.518585 12402 hybrid_clock.cc:648] HybridClock initialized: now 1786055980518507 us; error 68 us; skew 500 ppm
I20260806 22:39:40.519109 12402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:39:40.525606 12402 webserver.cc:460] Webserver started at http://127.15.198.130:37705/ using document root <none> and password file <none>
I20260806 22:39:40.526640 12402 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:40.526808 12402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:40.527298 12402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:40.531359 12402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/data/instance:
uuid: "54e1e4ef9835440597ec899af326adda"
format_stamp: "Formatted at 2026-08-06 22:39:40 on dist-test-slave-h3ft"
server_key: "dd130bb7873571217520c883a3ee35cc"
server_key_iv: "053878012bd91f22e1c6cd548346f7fd"
server_key_version: "encryptionkey@0"
I20260806 22:39:40.532395 12402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance:
uuid: "54e1e4ef9835440597ec899af326adda"
format_stamp: "Formatted at 2026-08-06 22:39:40 on dist-test-slave-h3ft"
server_key: "dd130bb7873571217520c883a3ee35cc"
server_key_iv: "053878012bd91f22e1c6cd548346f7fd"
server_key_version: "encryptionkey@0"
I20260806 22:39:40.539435 12402 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260806 22:39:40.544210 12419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:40.546350 12402 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:39:40.546576 12402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/wal
uuid: "54e1e4ef9835440597ec899af326adda"
format_stamp: "Formatted at 2026-08-06 22:39:40 on dist-test-slave-h3ft"
server_key: "dd130bb7873571217520c883a3ee35cc"
server_key_iv: "053878012bd91f22e1c6cd548346f7fd"
server_key_version: "encryptionkey@0"
I20260806 22:39:40.546856 12402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:40.589133 12402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:40.590342 12402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:40.590756 12402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:40.592048 12402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:40.593971 12402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:40.594111 12402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:40.594267 12402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:40.594364 12402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:40.634189 12402 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.130:38703
I20260806 22:39:40.634249 12569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.130:38703 every 8 connection(s)
I20260806 22:39:40.635953 12402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/data/info.pb
I20260806 22:39:40.638137 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 12402
I20260806 22:39:40.638360 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/wal/instance
I20260806 22:39:40.638885 16154 external_mini_cluster.cc:1428] Setting key f739219dad1f5b0b5f0ae2a989c41fe6
I20260806 22:39:40.643085 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.198.131:0
--local_ip_for_outbound_sockets=127.15.198.131
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:46555
--builtin_ntp_servers=127.15.198.148:36509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:39:40.652326 12575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:40.639564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:54929 (local address 127.15.198.190:46555)
0806 22:39:40.640193 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:40.640201 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:40.642009 (+ 1808us) server_negotiation.cc:407] Connection header received
0806 22:39:40.644294 (+ 2285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:40.644300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:40.644412 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:40.644546 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:40.646026 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:40.647056 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:40.648842 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:40.649316 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:40.649465 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:40.651310 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:40.651316 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:40.651322 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:40.651355 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:40.651407 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:40.651421 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:40.651549 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:40.651781 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 22:39:40.651905 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":293,"threads_started":1}
I20260806 22:39:40.653728 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:40.640416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:46555 (local address 127.15.198.130:54929)
0806 22:39:40.641741 (+ 1325us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:40.641866 (+ 125us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:40.643960 (+ 2094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:40.644832 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:40.644842 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:40.644957 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:40.645828 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:40.645853 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:40.647235 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:40.647240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:40.648683 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:40.648697 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:40.649590 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:40.649599 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:40.649867 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:40.650871 (+ 1004us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:39:40.650877 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:40.650939 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:39:40.651091 (+ 152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:40.651526 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:40.651539 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:40.651549 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 22:39:40.651769 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 22:39:40.652265 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":415,"threads_started":1}
I20260806 22:39:40.655618 12571 heartbeater.cc:347] Connected to a master server at 127.15.198.190:46555
I20260806 22:39:40.656069 12571 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:40.657349 12571 heartbeater.cc:511] Master 127.15.198.190:46555 requested a full tablet report, sending...
I20260806 22:39:40.659570 12144 ts_manager.cc:194] Registered new tserver with Master: 54e1e4ef9835440597ec899af326adda (127.15.198.130:38703)
I20260806 22:39:40.660658 12144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.198.130:54929
I20260806 22:39:41.054962 12396 heartbeater.cc:503] Master 127.15.198.190:46555 was elected leader, sending a full tablet report...
W20260806 22:39:41.078756 12577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:41.079248 12577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:41.079442 12577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:41.087674 12577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:41.087874 12577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:41.088079 12577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.198.131
I20260806 22:39:41.098536 12577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:36509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.198.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.198.131
--webserver_port=0
--tserver_master_addrs=127.15.198.190:46555
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.198.131
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:41.100507 12577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:41.103039 12577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:41.119470 12590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:41.119553 12587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:41.119603 12586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:41.120891 12577 server_base.cc:1034] running on GCE node
I20260806 22:39:41.121762 12577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:41.123144 12577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:41.124387 12577 hybrid_clock.cc:648] HybridClock initialized: now 1786055981124338 us; error 51 us; skew 500 ppm
I20260806 22:39:41.124811 12577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 22:39:41.128170 12577 webserver.cc:460] Webserver started at http://127.15.198.131:43765/ using document root <none> and password file <none>
I20260806 22:39:41.129199 12577 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:41.129340 12577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:41.129819 12577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:41.133776 12577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/data/instance:
uuid: "3eb1f971ef364a96876487e5ae75cb51"
format_stamp: "Formatted at 2026-08-06 22:39:41 on dist-test-slave-h3ft"
server_key: "a47f0c8657d1e4724a99d8f63f7cb10e"
server_key_iv: "ea38eb9a0bc551ff148cbdc8b59fc716"
server_key_version: "encryptionkey@0"
I20260806 22:39:41.134838 12577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance:
uuid: "3eb1f971ef364a96876487e5ae75cb51"
format_stamp: "Formatted at 2026-08-06 22:39:41 on dist-test-slave-h3ft"
server_key: "a47f0c8657d1e4724a99d8f63f7cb10e"
server_key_iv: "ea38eb9a0bc551ff148cbdc8b59fc716"
server_key_version: "encryptionkey@0"
I20260806 22:39:41.141503 12577 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 22:39:41.146275 12598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:41.148269 12577 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 22:39:41.148510 12577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/wal
uuid: "3eb1f971ef364a96876487e5ae75cb51"
format_stamp: "Formatted at 2026-08-06 22:39:41 on dist-test-slave-h3ft"
server_key: "a47f0c8657d1e4724a99d8f63f7cb10e"
server_key_iv: "ea38eb9a0bc551ff148cbdc8b59fc716"
server_key_version: "encryptionkey@0"
I20260806 22:39:41.148761 12577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:41.198017 12577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:41.199209 12577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:41.199594 12577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:41.200870 12577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:41.202783 12577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:41.202885 12577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:41.203033 12577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:41.203140 12577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 22:39:41.244907 12577 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.131:44779
I20260806 22:39:41.244957 12757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.131:44779 every 8 connection(s)
I20260806 22:39:41.246698 12577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/data/info.pb
I20260806 22:39:41.247282 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 12577
I20260806 22:39:41.247560 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/wal/instance
I20260806 22:39:41.248246 16154 external_mini_cluster.cc:1428] Setting key 8e5526ac7dfbce5860b3f2dc15569b24
I20260806 22:39:41.263801 12760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:41.250344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.131:37585 (local address 127.15.198.190:46555)
0806 22:39:41.251319 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:41.251328 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:41.252937 (+ 1609us) server_negotiation.cc:407] Connection header received
0806 22:39:41.255793 (+ 2856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:41.255799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:41.255928 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:41.256077 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:41.257477 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.258532 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.260421 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.260888 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.261021 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:41.262651 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:41.262656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:41.262659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:41.262685 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:41.262742 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:41.262756 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:41.262835 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:41.263185 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 22:39:41.263294 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":641,"threads_started":1}
I20260806 22:39:41.265357 12761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:41.251367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:46555 (local address 127.15.198.131:37585)
0806 22:39:41.252764 (+ 1397us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:41.252822 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:41.255425 (+ 2603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:41.256351 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:41.256362 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:41.256490 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:41.257249 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:41.257270 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.258717 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.258722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:41.260244 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:41.260258 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.261127 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.261131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:41.261392 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:41.262245 (+ 853us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:39:41.262248 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:41.262301 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:39:41.262416 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:41.262874 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:41.262888 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:41.262897 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 22:39:41.263152 (+ 255us) client_negotiation.cc:240] Negotiation successful
0806 22:39:41.263688 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":383,"threads_started":1}
I20260806 22:39:41.267186 12758 heartbeater.cc:347] Connected to a master server at 127.15.198.190:46555
I20260806 22:39:41.267661 12758 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:41.268801 12758 heartbeater.cc:511] Master 127.15.198.190:46555 requested a full tablet report, sending...
I20260806 22:39:41.270730 12144 ts_manager.cc:194] Registered new tserver with Master: 3eb1f971ef364a96876487e5ae75cb51 (127.15.198.131:44779)
I20260806 22:39:41.271793 12144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.198.131:37585
I20260806 22:39:41.278784 16154 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:39:41.291612 12760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:41.284096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49548 (local address 127.15.198.190:46555)
0806 22:39:41.284322 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:41.284330 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:41.284809 (+ 479us) server_negotiation.cc:407] Connection header received
0806 22:39:41.284993 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:41.284999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:41.285092 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:41.285209 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:41.285736 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.286959 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.288476 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.289177 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.289311 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:41.290825 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:41.290830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:41.290834 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:41.290857 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:41.290903 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:41.290917 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:41.290995 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:41.291253 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 22:39:41.291351 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 22:39:41.296723 12144 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:39:41.304926 12144 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:39:41.352864 12775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:41.321572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.15.198.129:33845)
0806 22:39:41.341698 (+ 20126us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:41.341791 (+ 93us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:41.341843 (+ 52us) server_negotiation.cc:407] Connection header received
0806 22:39:41.341969 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:41.341978 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:41.342238 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:41.342486 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:41.343887 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.345733 (+ 1846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.349324 (+ 3591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.350979 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.351310 (+ 331us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:41.351628 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:41.351926 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 22:39:41.352149 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19882,"thread_start_us":19622,"threads_started":1}
I20260806 22:39:41.353801 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:41.325832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47230 (local address 127.15.198.131:44779)
0806 22:39:41.339266 (+ 13434us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:41.339284 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:41.341439 (+ 2155us) server_negotiation.cc:407] Connection header received
0806 22:39:41.341619 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:41.341633 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:41.342022 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:41.342351 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:41.343475 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.345491 (+ 2016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.349266 (+ 3775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.350948 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.351308 (+ 360us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:41.351630 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:41.353005 (+ 1375us) server_negotiation.cc:299] Negotiation successful
0806 22:39:41.353348 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13107,"thread_start_us":12858,"threads_started":1}
I20260806 22:39:41.354094 12777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:41.325184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:44779 (local address 127.0.0.1:47230)
0806 22:39:41.341035 (+ 15851us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:41.341061 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:41.341243 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:41.342365 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:41.342375 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:41.342424 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:41.343074 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:41.343095 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.345676 (+ 2581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.345683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:41.348978 (+ 3295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:41.349001 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.351791 (+ 2790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.351800 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:41.352002 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:41.352050 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 22:39:41.352151 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:39:41.352328 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15710,"mutex_wait_us":126,"thread_start_us":14399,"threads_started":1}
I20260806 22:39:41.354164 12776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:41.321571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:33845 (local address 127.0.0.1:48068)
0806 22:39:41.336620 (+ 15049us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:41.336709 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:41.337085 (+ 376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:41.342831 (+ 5746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:41.342842 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:41.342875 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:41.343573 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:41.343597 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.345928 (+ 2331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.345934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:41.349013 (+ 3079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:41.349032 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.351239 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.351245 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:41.351561 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:41.351636 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 22:39:41.351854 (+ 218us) client_negotiation.cc:240] Negotiation successful
0806 22:39:41.352121 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14490,"mutex_wait_us":9,"spinlock_wait_cycles":9856,"thread_start_us":15863,"threads_started":1}
I20260806 22:39:41.357012 12780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:41.321691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.130:38703 (local address 127.0.0.1:60110)
0806 22:39:41.340070 (+ 18379us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:41.340106 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:41.340337 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:41.342973 (+ 2636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:41.342979 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:41.343101 (+ 122us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:41.343790 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:41.343808 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.346928 (+ 3120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.346934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:41.350313 (+ 3379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:41.350332 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.355899 (+ 5567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.355912 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:41.356163 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:41.356219 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 22:39:41.356339 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 22:39:41.356522 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18152,"spinlock_wait_cycles":3456,"thread_start_us":19036,"threads_started":1}
I20260806 22:39:41.357084 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:41.322007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60110 (local address 127.15.198.130:38703)
0806 22:39:41.342206 (+ 20199us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:41.342244 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:41.342284 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:39:41.342399 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:41.342407 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:41.342685 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:41.342944 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:41.344832 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.346692 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.350572 (+ 3880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.352497 (+ 1925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.352743 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:41.353059 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:41.356444 (+ 3385us) server_negotiation.cc:299] Negotiation successful
0806 22:39:41.356660 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20008,"thread_start_us":19835,"threads_started":1}
I20260806 22:39:41.359302 12312 tablet_service.cc:1467] Processing CreateTablet for tablet 9aef0a0fedd4457eaec54dfbf3b65fee (DEFAULT_TABLE table=test-table [id=e2db268526084e11a315f2c93bf31ca3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:41.359306 12673 tablet_service.cc:1467] Processing CreateTablet for tablet 9aef0a0fedd4457eaec54dfbf3b65fee (DEFAULT_TABLE table=test-table [id=e2db268526084e11a315f2c93bf31ca3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:41.361208 12673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aef0a0fedd4457eaec54dfbf3b65fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:41.361202 12312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aef0a0fedd4457eaec54dfbf3b65fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:41.361675 12487 tablet_service.cc:1467] Processing CreateTablet for tablet 9aef0a0fedd4457eaec54dfbf3b65fee (DEFAULT_TABLE table=test-table [id=e2db268526084e11a315f2c93bf31ca3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:41.363426 12487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aef0a0fedd4457eaec54dfbf3b65fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:41.376859 12782 tablet_bootstrap.cc:492] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e: Bootstrap starting.
I20260806 22:39:41.381446 12785 tablet_bootstrap.cc:492] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda: Bootstrap starting.
I20260806 22:39:41.383526 12783 tablet_bootstrap.cc:492] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51: Bootstrap starting.
I20260806 22:39:41.384593 12782 tablet_bootstrap.cc:654] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:41.386441 12782 log.cc:824] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:41.386551 12785 tablet_bootstrap.cc:654] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:41.386935 12783 tablet_bootstrap.cc:654] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:41.388131 12785 log.cc:824] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:41.388391 12783 log.cc:824] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:41.392091 12785 tablet_bootstrap.cc:492] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda: No bootstrap required, opened a new log
I20260806 22:39:41.392264 12782 tablet_bootstrap.cc:492] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e: No bootstrap required, opened a new log
I20260806 22:39:41.392496 12785 ts_tablet_manager.cc:1397] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260806 22:39:41.392678 12782 ts_tablet_manager.cc:1397] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.006s
I20260806 22:39:41.394299 12783 tablet_bootstrap.cc:492] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51: No bootstrap required, opened a new log
I20260806 22:39:41.394634 12783 ts_tablet_manager.cc:1397] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260806 22:39:41.402033 12782 raft_consensus.cc:348] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60336ea1ee054eec84058b6c0bdccf4e" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 33845 } } peers { permanent_uuid: "54e1e4ef9835440597ec899af326adda" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38703 } } peers { permanent_uuid: "3eb1f971ef364a96876487e5ae75cb51" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44779 } }
I20260806 22:39:41.402611 12782 raft_consensus.cc:374] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:41.402745 12782 raft_consensus.cc:729] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60336ea1ee054eec84058b6c0bdccf4e, State: Initialized, Role: FOLLOWER
I20260806 22:39:41.403885 12782 consensus_queue.cc:260] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60336ea1ee054eec84058b6c0bdccf4e" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 33845 } } peers { permanent_uuid: "54e1e4ef9835440597ec899af326adda" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38703 } } peers { permanent_uuid: "3eb1f971ef364a96876487e5ae75cb51" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44779 } }
I20260806 22:39:41.404318 12785 raft_consensus.cc:348] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60336ea1ee054eec84058b6c0bdccf4e" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 33845 } } peers { permanent_uuid: "54e1e4ef9835440597ec899af326adda" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38703 } } peers { permanent_uuid: "3eb1f971ef364a96876487e5ae75cb51" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44779 } }
I20260806 22:39:41.404848 12785 raft_consensus.cc:374] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:41.404946 12785 raft_consensus.cc:729] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54e1e4ef9835440597ec899af326adda, State: Initialized, Role: FOLLOWER
I20260806 22:39:41.405776 12785 consensus_queue.cc:260] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60336ea1ee054eec84058b6c0bdccf4e" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 33845 } } peers { permanent_uuid: "54e1e4ef9835440597ec899af326adda" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38703 } } peers { permanent_uuid: "3eb1f971ef364a96876487e5ae75cb51" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44779 } }
I20260806 22:39:41.406412 12783 raft_consensus.cc:348] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60336ea1ee054eec84058b6c0bdccf4e" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 33845 } } peers { permanent_uuid: "54e1e4ef9835440597ec899af326adda" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38703 } } peers { permanent_uuid: "3eb1f971ef364a96876487e5ae75cb51" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44779 } }
I20260806 22:39:41.407055 12783 raft_consensus.cc:374] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:41.407192 12783 raft_consensus.cc:729] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb1f971ef364a96876487e5ae75cb51, State: Initialized, Role: FOLLOWER
I20260806 22:39:41.408175 12782 ts_tablet_manager.cc:1428] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20260806 22:39:41.407953 12783 consensus_queue.cc:260] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60336ea1ee054eec84058b6c0bdccf4e" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 33845 } } peers { permanent_uuid: "54e1e4ef9835440597ec899af326adda" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38703 } } peers { permanent_uuid: "3eb1f971ef364a96876487e5ae75cb51" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44779 } }
I20260806 22:39:41.409621 12571 heartbeater.cc:503] Master 127.15.198.190:46555 was elected leader, sending a full tablet report...
I20260806 22:39:41.410595 12785 ts_tablet_manager.cc:1428] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda: Time spent starting tablet: real 0.018s user 0.014s sys 0.004s
I20260806 22:39:41.412084 12758 heartbeater.cc:503] Master 127.15.198.190:46555 was elected leader, sending a full tablet report...
I20260806 22:39:41.413610 12783 ts_tablet_manager.cc:1428] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51: Time spent starting tablet: real 0.019s user 0.015s sys 0.005s
W20260806 22:39:41.499249 12759 tablet.cc:2369] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:39:41.499483 12790 raft_consensus.cc:482] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:39:41.499974 12790 raft_consensus.cc:504] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60336ea1ee054eec84058b6c0bdccf4e" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 33845 } } peers { permanent_uuid: "54e1e4ef9835440597ec899af326adda" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38703 } } peers { permanent_uuid: "3eb1f971ef364a96876487e5ae75cb51" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44779 } }
I20260806 22:39:41.502334 12790 leader_election.cc:288] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60336ea1ee054eec84058b6c0bdccf4e (127.15.198.129:33845), 3eb1f971ef364a96876487e5ae75cb51 (127.15.198.131:44779)
I20260806 22:39:41.511055 12775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:41.502832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:34835 (local address 127.15.198.129:33845)
0806 22:39:41.503151 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:41.503160 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:41.504217 (+ 1057us) server_negotiation.cc:407] Connection header received
0806 22:39:41.504542 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:41.504547 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:41.504629 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:41.504757 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:41.505568 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.506306 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.508862 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.509790 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.509942 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:41.510010 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:41.510540 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 22:39:41.510699 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 22:39:41.511067 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:41.502895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.198.130:41829 (local address 127.15.198.131:44779)
0806 22:39:41.503265 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:41.503276 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:41.504098 (+ 822us) server_negotiation.cc:407] Connection header received
0806 22:39:41.504494 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:41.504504 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:41.504600 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:41.504683 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:41.505425 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.506331 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.508948 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.509943 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.510143 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:41.510232 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:41.510539 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 22:39:41.510705 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260806 22:39:41.511183 12793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:41.502686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.131:44779 (local address 127.15.198.130:41829)
0806 22:39:41.503881 (+ 1195us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:41.503907 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:41.504268 (+ 361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:41.504740 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:41.504745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:41.504782 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:41.505244 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:41.505256 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.506508 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.506511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:41.508700 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:41.508713 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.510165 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.510169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:41.510302 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:41.510380 (+ 78us) client_negotiation.cc:769] Sending connection context
0806 22:39:41.510497 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 22:39:41.510680 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"mutex_wait_us":154,"thread_start_us":905,"threads_started":1}
I20260806 22:39:41.511487 12792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:41.502833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.129:33845 (local address 127.15.198.130:34835)
0806 22:39:41.504035 (+ 1202us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:41.504066 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:41.504337 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:41.504807 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:41.504813 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:41.504846 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:41.505376 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:41.505391 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.506472 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.506477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:41.508629 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:41.508643 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.510038 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.510049 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:41.510300 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:41.510381 (+ 81us) client_negotiation.cc:769] Sending connection context
0806 22:39:41.510502 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 22:39:41.510682 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1058,"mutex_wait_us":128,"spinlock_wait_cycles":36224,"thread_start_us":583,"threads_started":1}
I20260806 22:39:41.512953 12698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aef0a0fedd4457eaec54dfbf3b65fee" candidate_uuid: "54e1e4ef9835440597ec899af326adda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb1f971ef364a96876487e5ae75cb51" is_pre_election: true
I20260806 22:39:41.512953 12336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aef0a0fedd4457eaec54dfbf3b65fee" candidate_uuid: "54e1e4ef9835440597ec899af326adda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60336ea1ee054eec84058b6c0bdccf4e" is_pre_election: true
I20260806 22:39:41.513824 12698 raft_consensus.cc:2454] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54e1e4ef9835440597ec899af326adda in term 0.
I20260806 22:39:41.513825 12336 raft_consensus.cc:2454] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54e1e4ef9835440597ec899af326adda in term 0.
I20260806 22:39:41.514919 12423 leader_election.cc:302] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3eb1f971ef364a96876487e5ae75cb51, 54e1e4ef9835440597ec899af326adda, 60336ea1ee054eec84058b6c0bdccf4e; no voters:
I20260806 22:39:41.515573 12790 raft_consensus.cc:2781] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:39:41.515782 12790 raft_consensus.cc:482] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:39:41.515918 12790 raft_consensus.cc:3037] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:41.518816 12790 raft_consensus.cc:504] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60336ea1ee054eec84058b6c0bdccf4e" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 33845 } } peers { permanent_uuid: "54e1e4ef9835440597ec899af326adda" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38703 } } peers { permanent_uuid: "3eb1f971ef364a96876487e5ae75cb51" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44779 } }
I20260806 22:39:41.519657 12790 leader_election.cc:288] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [CANDIDATE]: Term 1 election: Requested vote from peers 60336ea1ee054eec84058b6c0bdccf4e (127.15.198.129:33845), 3eb1f971ef364a96876487e5ae75cb51 (127.15.198.131:44779)
I20260806 22:39:41.520325 12698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aef0a0fedd4457eaec54dfbf3b65fee" candidate_uuid: "54e1e4ef9835440597ec899af326adda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb1f971ef364a96876487e5ae75cb51"
I20260806 22:39:41.520331 12336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aef0a0fedd4457eaec54dfbf3b65fee" candidate_uuid: "54e1e4ef9835440597ec899af326adda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60336ea1ee054eec84058b6c0bdccf4e"
I20260806 22:39:41.520609 12336 raft_consensus.cc:3037] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:41.520704 12698 raft_consensus.cc:3037] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51 [term 0 FOLLOWER]: Advancing to term 1
W20260806 22:39:41.523275 12397 tablet.cc:2369] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:39:41.524418 12698 raft_consensus.cc:2454] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54e1e4ef9835440597ec899af326adda in term 1.
I20260806 22:39:41.524539 12336 raft_consensus.cc:2454] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54e1e4ef9835440597ec899af326adda in term 1.
I20260806 22:39:41.525110 12423 leader_election.cc:302] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb1f971ef364a96876487e5ae75cb51, 54e1e4ef9835440597ec899af326adda; no voters:
I20260806 22:39:41.525631 12790 raft_consensus.cc:2781] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:41.526345 12790 raft_consensus.cc:686] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [term 1 LEADER]: Becoming Leader. State: Replica: 54e1e4ef9835440597ec899af326adda, State: Running, Role: LEADER
I20260806 22:39:41.527169 12790 consensus_queue.cc:237] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60336ea1ee054eec84058b6c0bdccf4e" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 33845 } } peers { permanent_uuid: "54e1e4ef9835440597ec899af326adda" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38703 } } peers { permanent_uuid: "3eb1f971ef364a96876487e5ae75cb51" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44779 } }
I20260806 22:39:41.535036 12144 catalog_manager.cc:5495] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda reported cstate change: term changed from 0 to 1, leader changed from <none> to 54e1e4ef9835440597ec899af326adda (127.15.198.130). New cstate: current_term: 1 leader_uuid: "54e1e4ef9835440597ec899af326adda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60336ea1ee054eec84058b6c0bdccf4e" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 33845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54e1e4ef9835440597ec899af326adda" member_type: VOTER last_known_addr { host: "127.15.198.130" port: 38703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3eb1f971ef364a96876487e5ae75cb51" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44779 } health_report { overall_health: UNKNOWN } } }
I20260806 22:39:41.599550 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:41.591804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60126 (local address 127.15.198.130:38703)
0806 22:39:41.592079 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:41.592084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:41.592203 (+ 119us) server_negotiation.cc:407] Connection header received
0806 22:39:41.592491 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:41.592496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:41.592633 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:41.592804 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:39:41.593332 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.594128 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.596628 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.597098 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.597207 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:41.597966 (+ 759us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:39:41.598699 (+ 733us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:39:41.598831 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:41.598996 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 22:39:41.599170 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 22:39:41.613413 12698 raft_consensus.cc:1261] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51 [term 1 FOLLOWER]: Refusing update from remote peer 54e1e4ef9835440597ec899af326adda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:41.613413 12336 raft_consensus.cc:1261] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e [term 1 FOLLOWER]: Refusing update from remote peer 54e1e4ef9835440597ec899af326adda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:41.614917 12795 consensus_queue.cc:1035] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [LEADER]: Connected to new peer: Peer: permanent_uuid: "60336ea1ee054eec84058b6c0bdccf4e" member_type: VOTER last_known_addr { host: "127.15.198.129" port: 33845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:41.615427 12790 consensus_queue.cc:1035] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb1f971ef364a96876487e5ae75cb51" member_type: VOTER last_known_addr { host: "127.15.198.131" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:41.632006 12798 mvcc.cc:205] Tried to move back new op lower bound from 7315685300672929792 to 7315685300341891072. Current Snapshot: MvccSnapshot[applied={T|T < 7315685300672929792}]
I20260806 22:39:41.663828 12144 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:39:41.664228 12144 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:39:41.670150 12144 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b0063735de34357963d9fbf96ec110a: Sending DeleteTablet for 3 replicas of tablet 9aef0a0fedd4457eaec54dfbf3b65fee
I20260806 22:39:41.672353 12487 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aef0a0fedd4457eaec54dfbf3b65fee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:41 UTC) from {username='slave'} at 127.0.0.1:60110
I20260806 22:39:41.674059 12312 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aef0a0fedd4457eaec54dfbf3b65fee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:41 UTC) from {username='slave'} at 127.0.0.1:48068
I20260806 22:39:41.674329 12673 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aef0a0fedd4457eaec54dfbf3b65fee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:41 UTC) from {username='slave'} at 127.0.0.1:47230
I20260806 22:39:41.675349 12814 tablet_replica.cc:330] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e: stopping tablet replica
I20260806 22:39:41.675989 12814 raft_consensus.cc:2229] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:39:41.676239 12816 tablet_replica.cc:330] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51: stopping tablet replica
I20260806 22:39:41.676703 12814 raft_consensus.cc:2258] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:41.676703 12816 raft_consensus.cc:2229] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:39:41.677116 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 12225
I20260806 22:39:41.677224 12816 raft_consensus.cc:2258] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:41.680253 12816 ts_tablet_manager.cc:1905] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:41.683188 12814 ts_tablet_manager.cc:1905] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:41.688593 12816 ts_tablet_manager.cc:1918] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:41.688846 12816 log.cc:1196] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-2/wal/wals/9aef0a0fedd4457eaec54dfbf3b65fee
I20260806 22:39:41.689487 12816 ts_tablet_manager.cc:1939] T 9aef0a0fedd4457eaec54dfbf3b65fee P 3eb1f971ef364a96876487e5ae75cb51: Deleting consensus metadata
I20260806 22:39:41.691376 12814 ts_tablet_manager.cc:1918] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:41.691586 12814 log.cc:1196] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-0/wal/wals/9aef0a0fedd4457eaec54dfbf3b65fee
I20260806 22:39:41.692229 12814 ts_tablet_manager.cc:1939] T 9aef0a0fedd4457eaec54dfbf3b65fee P 60336ea1ee054eec84058b6c0bdccf4e: Deleting consensus metadata
I20260806 22:39:41.692353 12126 catalog_manager.cc:4841] TS 3eb1f971ef364a96876487e5ae75cb51 (127.15.198.131:44779): tablet 9aef0a0fedd4457eaec54dfbf3b65fee (table test-table [id=e2db268526084e11a315f2c93bf31ca3]) successfully deleted
I20260806 22:39:41.693033 12812 tablet_replica.cc:330] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda: stopping tablet replica
I20260806 22:39:41.693799 12812 raft_consensus.cc:2229] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:39:41.694432 12125 catalog_manager.cc:4841] TS 60336ea1ee054eec84058b6c0bdccf4e (127.15.198.129:33845): tablet 9aef0a0fedd4457eaec54dfbf3b65fee (table test-table [id=e2db268526084e11a315f2c93bf31ca3]) successfully deleted
I20260806 22:39:41.694823 12812 raft_consensus.cc:2258] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:41.697299 12812 ts_tablet_manager.cc:1905] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:41.704572 12775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:41.678770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48072 (local address 127.15.198.129:33845)
0806 22:39:41.678986 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:41.678993 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:41.695163 (+ 16170us) server_negotiation.cc:407] Connection header received
0806 22:39:41.695525 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:41.695531 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:41.695663 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:41.695756 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:41.696590 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.697739 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.700014 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.700696 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.700836 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:41.702433 (+ 1597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:41.702455 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:41.702478 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:41.703518 (+ 1040us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:41.703601 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:41.703639 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:41.703765 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:41.703958 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 22:39:41.704121 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 22:39:41.705482 12390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:41.705593 12812 ts_tablet_manager.cc:1918] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:41.705848 12812 log.cc:1196] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda: Deleting WAL directory at /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055595907839-16154-0/minicluster-data/ts-1/wal/wals/9aef0a0fedd4457eaec54dfbf3b65fee
I20260806 22:39:41.706681 12812 ts_tablet_manager.cc:1939] T 9aef0a0fedd4457eaec54dfbf3b65fee P 54e1e4ef9835440597ec899af326adda: Deleting consensus metadata
I20260806 22:39:41.709724 12127 catalog_manager.cc:4841] TS 54e1e4ef9835440597ec899af326adda (127.15.198.130:38703): tablet 9aef0a0fedd4457eaec54dfbf3b65fee (table test-table [id=e2db268526084e11a315f2c93bf31ca3]) successfully deleted
I20260806 22:39:41.810546 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 12225
I20260806 22:39:41.852319 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 12402
I20260806 22:39:41.861301 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:41.853598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60130 (local address 127.15.198.130:38703)
0806 22:39:41.853858 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:41.853865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:41.853894 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:39:41.853974 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:41.853978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:41.854080 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:41.854221 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:41.854757 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.855881 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.857239 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:41.857774 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:41.857931 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:41.859459 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:41.859471 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:41.859496 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:41.860326 (+ 830us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:41.860392 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:41.860415 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:41.860568 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:41.860714 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 22:39:41.860857 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 22:39:41.862291 12563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:42.002264 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 12402
I20260806 22:39:42.034612 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 12577
I20260806 22:39:42.043644 12822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:42.035675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47232 (local address 127.15.198.131:44779)
0806 22:39:42.036245 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:42.036250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:42.036272 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:39:42.036348 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:42.036353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:42.036458 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:42.036567 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:42.037038 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:42.037831 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:42.039450 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:42.040045 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:42.040182 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:42.041642 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:42.041654 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:42.041675 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:42.042555 (+ 880us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:42.042632 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:42.042661 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:42.042790 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:42.042923 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 22:39:42.043109 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":295,"threads_started":1}
I20260806 22:39:42.044627 12752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:42.144349 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 12577
I20260806 22:39:42.174624 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 12104
I20260806 22:39:42.176393 12190 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:42.310119 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 12104
2026-08-06T22:39:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3716 ms)
[----------] 6 tests from EncryptionPolicyTest (28526 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:39:42 dist-test-slave-h3ft krb5kdc[12828](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:39:42 dist-test-slave-h3ft krb5kdc[12828](info): set up 2 sockets
Aug 06 22:39:42 dist-test-slave-h3ft krb5kdc[12828](info): commencing operation
krb5kdc: starting...
W20260806 22:39:43.772432 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.402s user 0.000s sys 0.070s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:39:44 dist-test-slave-h3ft krb5kdc[12828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:39:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:39:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:39:44.220121 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35753
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:44.661103 12847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:44.661551 12847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:44.661662 12847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:44.670198 12847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:44.670372 12847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:44.670437 12847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:44.670501 12847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:44.670552 12847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:44.682216 12847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35753
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:44.684064 12847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:44.685906 12847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:44.700953 12854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:44.700789 12858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:44.700860 12856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:44.702272 12847 server_base.cc:1034] running on GCE node
I20260806 22:39:44.703554 12847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:44.705651 12847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:44.706954 12847 hybrid_clock.cc:648] HybridClock initialized: now 1786055984706898 us; error 55 us; skew 500 ppm
I20260806 22:39:44.707468 12847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:44 dist-test-slave-h3ft krb5kdc[12828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055984, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:44.711588 12847 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:39:44.714066 12847 webserver.cc:460] Webserver started at http://127.0.0.1:41649/ using document root <none> and password file <none>
I20260806 22:39:44.715046 12847 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:44.715202 12847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:44.715595 12847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:44.719561 12847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "96da7016edbe472ab2af2f80801ef31d"
format_stamp: "Formatted at 2026-08-06 22:39:44 on dist-test-slave-h3ft"
server_key: "b921e8237322a0532b725c90e74ab00a"
server_key_iv: "a34b92a45d9f8388e0934dba0d9dec65"
server_key_version: "encryptionkey@0"
I20260806 22:39:44.720556 12847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "96da7016edbe472ab2af2f80801ef31d"
format_stamp: "Formatted at 2026-08-06 22:39:44 on dist-test-slave-h3ft"
server_key: "b921e8237322a0532b725c90e74ab00a"
server_key_iv: "a34b92a45d9f8388e0934dba0d9dec65"
server_key_version: "encryptionkey@0"
I20260806 22:39:44.727214 12847 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260806 22:39:44.731514 12867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:44.733270 12847 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:39:44.733518 12847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "96da7016edbe472ab2af2f80801ef31d"
format_stamp: "Formatted at 2026-08-06 22:39:44 on dist-test-slave-h3ft"
server_key: "b921e8237322a0532b725c90e74ab00a"
server_key_iv: "a34b92a45d9f8388e0934dba0d9dec65"
server_key_version: "encryptionkey@0"
I20260806 22:39:44.733742 12847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:44.769246 12847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:44.775732 12847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:44.776080 12847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:44.799538 12847 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35753
I20260806 22:39:44.799592 12937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35753 every 8 connection(s)
I20260806 22:39:44.801550 12847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:39:44.806839 12939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:44.807912 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 12847
I20260806 22:39:44.808180 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:39:44.808997 16154 external_mini_cluster.cc:1428] Setting key 930bc20959088a79015876bacd609a20
I20260806 22:39:44.825047 12939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d: Bootstrap starting.
Aug 06 22:39:44 dist-test-slave-h3ft krb5kdc[12828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:39:44.829720 12939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:44.831502 12939 log.cc:824] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:44.837370 12939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d: No bootstrap required, opened a new log
I20260806 22:39:44.842994 12942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:44.811739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36885 (local address 127.0.0.1:35753)
0806 22:39:44.812770 (+ 1031us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:44.812790 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:44.812856 (+ 66us) server_negotiation.cc:407] Connection header received
0806 22:39:44.815299 (+ 2443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:44.815341 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:44.816090 (+ 749us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:44.816683 (+ 593us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:44.818800 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:44.821227 (+ 2427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:44.823072 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:44.824284 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:44.824506 (+ 222us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:44.830584 (+ 6078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:44.830599 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:44.830625 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:44.830698 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:44.836155 (+ 5457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:44.837901 (+ 1746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:44.837909 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:44.837918 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:44.838193 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:44.838855 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:44.838862 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:44.838865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:44.839738 (+ 873us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:44.840186 (+ 448us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:44.840808 (+ 622us) server_negotiation.cc:299] Negotiation successful
0806 22:39:44.841260 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":426,"threads_started":1}
I20260806 22:39:44.851177 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:39:44.850896 12939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96da7016edbe472ab2af2f80801ef31d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } }
I20260806 22:39:44.851706 12939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:44.851828 12939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96da7016edbe472ab2af2f80801ef31d, State: Initialized, Role: FOLLOWER
I20260806 22:39:44.852747 12939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96da7016edbe472ab2af2f80801ef31d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } }
I20260806 22:39:44.853118 12939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:39:44.853281 12939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:39:44.853492 12939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:44.857872 12939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96da7016edbe472ab2af2f80801ef31d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } }
I20260806 22:39:44.858814 12939 leader_election.cc:302] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96da7016edbe472ab2af2f80801ef31d; no voters:
I20260806 22:39:44.859714 12939 leader_election.cc:288] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:39:44.859838 12954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:44.863381 12939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:44.867090 12954 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [term 1 LEADER]: Becoming Leader. State: Replica: 96da7016edbe472ab2af2f80801ef31d, State: Running, Role: LEADER
I20260806 22:39:44.868115 12954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96da7016edbe472ab2af2f80801ef31d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } }
I20260806 22:39:44.872622 12956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96da7016edbe472ab2af2f80801ef31d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96da7016edbe472ab2af2f80801ef31d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } } }
I20260806 22:39:44.873256 12956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:44.874918 12957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96da7016edbe472ab2af2f80801ef31d. Latest consensus state: current_term: 1 leader_uuid: "96da7016edbe472ab2af2f80801ef31d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96da7016edbe472ab2af2f80801ef31d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } } }
I20260806 22:39:44.875399 12957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:44.884959 12962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:39:44.886904 12942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:44.858864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38215 (local address 127.0.0.1:35753)
0806 22:39:44.859163 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:44.859171 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:44.863949 (+ 4778us) server_negotiation.cc:407] Connection header received
0806 22:39:44.864253 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:44.864259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:44.864350 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:44.864483 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:44.866520 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:44.867689 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:44.869347 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:44.869881 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:44.870048 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:44.874312 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:44.874324 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:44.874333 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:44.874412 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:44.880401 (+ 5989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:44.882107 (+ 1706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:44.882118 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:44.882125 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:44.882388 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:44.883230 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:44.883239 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:44.883244 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:44.884960 (+ 1716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:44.885384 (+ 424us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:44.886175 (+ 791us) server_negotiation.cc:299] Negotiation successful
0806 22:39:44.886450 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 22:39:44.902319 12962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:39:44.915092 12962 catalog_manager.cc:1347] Generated new cluster ID: d513f4adab5c46dfbd3d50c24d3ac46a
I20260806 22:39:44.915292 12962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:39:44.940078 12962 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:39:44.942441 12962 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:39:44.956297 12962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96da7016edbe472ab2af2f80801ef31d: Generated new TSK 0
I20260806 22:39:44.958081 12962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:39:44.970055 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 12847
I20260806 22:39:44.972924 12932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:45.081257 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 12847
W20260806 22:39:45.113044 12825 connection.cc:535] client connection to 127.0.0.1:35753 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35753): Connection reset by peer (error 104)
2026-08-06T22:39:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2765 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:39:45 dist-test-slave-h3ft krb5kdc[12982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:39:45 dist-test-slave-h3ft krb5kdc[12982](info): set up 2 sockets
Aug 06 22:39:45 dist-test-slave-h3ft krb5kdc[12982](info): commencing operation
krb5kdc: starting...
W20260806 22:39:46.522209 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.388s user 0.004s sys 0.064s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:39:46 dist-test-slave-h3ft krb5kdc[12982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:39:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:39:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:39:46.964157 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37969
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:47.403447 13001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:47.403872 13001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:47.404057 13001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:47.412575 13001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:47.412755 13001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:47.412866 13001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:47.413002 13001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:47.413102 13001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:47.425086 13001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37969
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:47.427126 13001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:47.428942 13001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:47.444233 13006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:47.444288 13009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:47.444320 13007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:47.445233 13001 server_base.cc:1034] running on GCE node
I20260806 22:39:47.446295 13001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:47.448230 13001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:47.449540 13001 hybrid_clock.cc:648] HybridClock initialized: now 1786055987449443 us; error 53 us; skew 500 ppm
I20260806 22:39:47.449990 13001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:47 dist-test-slave-h3ft krb5kdc[12982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055987, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:47.459108 13001 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:39:47.461494 13001 webserver.cc:460] Webserver started at http://127.0.0.1:40741/ using document root <none> and password file <none>
I20260806 22:39:47.462502 13001 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:47.462687 13001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:47.463172 13001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:47.467343 13001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "b57db22837894cb391ece762f5b7cd95"
format_stamp: "Formatted at 2026-08-06 22:39:47 on dist-test-slave-h3ft"
server_key: "21443c0fbbd031d0a181f2f62476874d"
server_key_iv: "7e3feb3e9b39b7067137c048781baa92"
server_key_version: "encryptionkey@0"
I20260806 22:39:47.468434 13001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "b57db22837894cb391ece762f5b7cd95"
format_stamp: "Formatted at 2026-08-06 22:39:47 on dist-test-slave-h3ft"
server_key: "21443c0fbbd031d0a181f2f62476874d"
server_key_iv: "7e3feb3e9b39b7067137c048781baa92"
server_key_version: "encryptionkey@0"
I20260806 22:39:47.476241 13001 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 22:39:47.481374 13022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:47.483403 13001 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:39:47.483662 13001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "b57db22837894cb391ece762f5b7cd95"
format_stamp: "Formatted at 2026-08-06 22:39:47 on dist-test-slave-h3ft"
server_key: "21443c0fbbd031d0a181f2f62476874d"
server_key_iv: "7e3feb3e9b39b7067137c048781baa92"
server_key_version: "encryptionkey@0"
I20260806 22:39:47.483955 13001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:47.509370 13001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:47.515990 13001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:47.516376 13001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:47.542393 13001 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37969
I20260806 22:39:47.542408 13087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37969 every 8 connection(s)
I20260806 22:39:47.544386 13001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:39:47.549438 13088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:47.552047 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 13001
I20260806 22:39:47.552301 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:39:47.552975 16154 external_mini_cluster.cc:1428] Setting key 0b6e162591fa1bfa8babd8dc0e5cad67
I20260806 22:39:47.564329 13088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95: Bootstrap starting.
I20260806 22:39:47.568488 13088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:47.570240 13088 log.cc:824] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:47.571079 13093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:47.555718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37733 (local address 127.0.0.1:37969)
0806 22:39:47.556851 (+ 1133us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:47.556899 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:47.556962 (+ 63us) server_negotiation.cc:407] Connection header received
0806 22:39:47.559179 (+ 2217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:47.559213 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:47.559797 (+ 584us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:47.560338 (+ 541us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:47.561129 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:47.562850 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:47.564571 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:47.565415 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:47.565647 (+ 232us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:47.567438 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:47.567449 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:47.567468 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:47.568190 (+ 722us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:47.568291 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:47.568323 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:47.568575 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:47.568998 (+ 423us) server_negotiation.cc:299] Negotiation successful
0806 22:39:47.569267 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":481,"threads_started":1}
I20260806 22:39:47.574646 13088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95: No bootstrap required, opened a new log
I20260806 22:39:47.576660 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:39:47.586833 13088 raft_consensus.cc:348] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57db22837894cb391ece762f5b7cd95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37969 } }
I20260806 22:39:47.587553 13088 raft_consensus.cc:374] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:47.587687 13088 raft_consensus.cc:729] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b57db22837894cb391ece762f5b7cd95, State: Initialized, Role: FOLLOWER
I20260806 22:39:47.588601 13088 consensus_queue.cc:260] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57db22837894cb391ece762f5b7cd95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37969 } }
I20260806 22:39:47.589083 13088 raft_consensus.cc:388] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:39:47.589287 13088 raft_consensus.cc:482] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:39:47.589536 13088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:47.591619 13093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:47.583175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47465 (local address 127.0.0.1:37969)
0806 22:39:47.583461 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:47.583472 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:47.584025 (+ 553us) server_negotiation.cc:407] Connection header received
0806 22:39:47.584259 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:47.584264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:47.584344 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:47.584470 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:47.585213 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:47.586093 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:47.588168 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:47.588923 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:47.589080 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:47.590583 (+ 1503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:47.590591 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:47.590596 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:47.590624 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:47.590709 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:47.590731 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:47.590865 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:47.591157 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 22:39:47.591234 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 22:39:47.593297 13088 raft_consensus.cc:504] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57db22837894cb391ece762f5b7cd95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37969 } }
I20260806 22:39:47.594054 13088 leader_election.cc:302] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b57db22837894cb391ece762f5b7cd95; no voters:
I20260806 22:39:47.594771 13088 leader_election.cc:288] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:39:47.594961 13109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:47.596621 13109 raft_consensus.cc:686] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [term 1 LEADER]: Becoming Leader. State: Replica: b57db22837894cb391ece762f5b7cd95, State: Running, Role: LEADER
I20260806 22:39:47.597314 13109 consensus_queue.cc:237] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57db22837894cb391ece762f5b7cd95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37969 } }
I20260806 22:39:47.597784 13088 sys_catalog.cc:564] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:47.601058 13115 sys_catalog.cc:455] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b57db22837894cb391ece762f5b7cd95. Latest consensus state: current_term: 1 leader_uuid: "b57db22837894cb391ece762f5b7cd95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57db22837894cb391ece762f5b7cd95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37969 } } }
I20260806 22:39:47.601090 13110 sys_catalog.cc:455] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b57db22837894cb391ece762f5b7cd95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57db22837894cb391ece762f5b7cd95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37969 } } }
I20260806 22:39:47.601675 13115 sys_catalog.cc:458] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:47.601740 13110 sys_catalog.cc:458] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95 [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:47.608039 13122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:39:47.614645 13122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:39:47.626655 13122 catalog_manager.cc:1347] Generated new cluster ID: 51003370bdb148c28e21854ce71deba6
I20260806 22:39:47.626843 13122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:39:47.639923 13122 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:39:47.642350 13122 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:39:47.651880 13122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b57db22837894cb391ece762f5b7cd95: Generated new TSK 0
I20260806 22:39:47.653498 13122 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:39:47.671816 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 13001
W20260806 22:39:47.673364 13082 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37733
W20260806 22:39:47.674345 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 13001: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:39:47.674589 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 13001
W20260806 22:39:47.707667 12979 connection.cc:535] client connection to 127.0.0.1:37969 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37969): Connection reset by peer (error 104)
2026-08-06T22:39:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2594 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:39:47 dist-test-slave-h3ft krb5kdc[13143](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:39:47 dist-test-slave-h3ft krb5kdc[13143](info): set up 2 sockets
Aug 06 22:39:47 dist-test-slave-h3ft krb5kdc[13143](info): commencing operation
krb5kdc: starting...
W20260806 22:39:49.092916 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.364s user 0.001s sys 0.066s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:39:49 dist-test-slave-h3ft krb5kdc[13143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:39:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:39:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:39:49.531680 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44445
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:49.978930 13159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:49.979424 13159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:49.979595 13159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:49.987881 13159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:49.988031 13159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:49.988144 13159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:49.988261 13159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:49.988335 13159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:50.000110 13159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44445
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:50.002120 13159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:50.003980 13159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:50.020726 13165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:50.020584 13168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:50.020692 13164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:50.022305 13159 server_base.cc:1034] running on GCE node
I20260806 22:39:50.023536 13159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:50.025506 13159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:50.026786 13159 hybrid_clock.cc:648] HybridClock initialized: now 1786055990026732 us; error 30 us; skew 500 ppm
I20260806 22:39:50.027345 13159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:50 dist-test-slave-h3ft krb5kdc[13143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055990, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:50.031718 13159 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:39:50.033877 13159 webserver.cc:460] Webserver started at http://127.0.0.1:35677/ using document root <none> and password file <none>
I20260806 22:39:50.034857 13159 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:50.034986 13159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:50.035403 13159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:50.039458 13159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "3b6399b339d8404cb5da05a7ae0a2fb0"
format_stamp: "Formatted at 2026-08-06 22:39:50 on dist-test-slave-h3ft"
server_key: "c1c1a1341898510d7ef4a6d688df78b5"
server_key_iv: "bbac9670d9c8fe7a5d4b40b615f18233"
server_key_version: "encryptionkey@0"
I20260806 22:39:50.040369 13159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "3b6399b339d8404cb5da05a7ae0a2fb0"
format_stamp: "Formatted at 2026-08-06 22:39:50 on dist-test-slave-h3ft"
server_key: "c1c1a1341898510d7ef4a6d688df78b5"
server_key_iv: "bbac9670d9c8fe7a5d4b40b615f18233"
server_key_version: "encryptionkey@0"
I20260806 22:39:50.047284 13159 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 22:39:50.052219 13176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:50.054220 13159 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:39:50.054518 13159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "3b6399b339d8404cb5da05a7ae0a2fb0"
format_stamp: "Formatted at 2026-08-06 22:39:50 on dist-test-slave-h3ft"
server_key: "c1c1a1341898510d7ef4a6d688df78b5"
server_key_iv: "bbac9670d9c8fe7a5d4b40b615f18233"
server_key_version: "encryptionkey@0"
I20260806 22:39:50.054836 13159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:50.062979 13159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:50.069162 13159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:50.069595 13159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:50.096256 13159 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44445
I20260806 22:39:50.096297 13250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44445 every 8 connection(s)
I20260806 22:39:50.098308 13159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:39:50.100463 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 13159
I20260806 22:39:50.100754 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:39:50.101646 16154 external_mini_cluster.cc:1428] Setting key ebeb8b1e32b27b2754de8cfca2f5529f
I20260806 22:39:50.105901 13251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:50.112475 13257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:50.104572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43515 (local address 127.0.0.1:44445)
0806 22:39:50.105697 (+ 1125us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:50.105714 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:50.105783 (+ 69us) server_negotiation.cc:407] Connection header received
0806 22:39:50.107946 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:50.107976 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:50.108436 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:50.108958 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:50.110323 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:50.110331 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:50.110346 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:50.110363 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:50.110458 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:50.110490 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:50.110782 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:50.110951 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 22:39:50.111231 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":818,"thread_start_us":493,"threads_started":1}
I20260806 22:39:50.117455 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:39:50.120379 13251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0: Bootstrap starting.
I20260806 22:39:50.125276 13251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:50.125907 13257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:50.122895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44999 (local address 127.0.0.1:44445)
0806 22:39:50.123183 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:50.123190 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:50.123526 (+ 336us) server_negotiation.cc:407] Connection header received
0806 22:39:50.123732 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:50.123735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:50.123798 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:50.123889 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:50.125100 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:50.125105 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:50.125109 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:50.125129 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:50.125176 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:50.125192 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:50.125293 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:50.125472 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 22:39:50.125551 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260806 22:39:50.126852 13251 log.cc:824] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:50.130489 13251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0: No bootstrap required, opened a new log
I20260806 22:39:50.139930 13251 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6399b339d8404cb5da05a7ae0a2fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44445 } }
I20260806 22:39:50.140488 13251 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:50.140635 13251 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b6399b339d8404cb5da05a7ae0a2fb0, State: Initialized, Role: FOLLOWER
I20260806 22:39:50.141412 13251 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6399b339d8404cb5da05a7ae0a2fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44445 } }
I20260806 22:39:50.141649 13251 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:39:50.141768 13251 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:39:50.141930 13251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:50.144521 13251 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6399b339d8404cb5da05a7ae0a2fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44445 } }
I20260806 22:39:50.145102 13251 leader_election.cc:302] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b6399b339d8404cb5da05a7ae0a2fb0; no voters:
I20260806 22:39:50.146015 13251 leader_election.cc:288] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:39:50.146279 13266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:50.147574 13266 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [term 1 LEADER]: Becoming Leader. State: Replica: 3b6399b339d8404cb5da05a7ae0a2fb0, State: Running, Role: LEADER
I20260806 22:39:50.148353 13266 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6399b339d8404cb5da05a7ae0a2fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44445 } }
I20260806 22:39:50.148993 13251 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:50.152014 13267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b6399b339d8404cb5da05a7ae0a2fb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6399b339d8404cb5da05a7ae0a2fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44445 } } }
I20260806 22:39:50.152127 13269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b6399b339d8404cb5da05a7ae0a2fb0. Latest consensus state: current_term: 1 leader_uuid: "3b6399b339d8404cb5da05a7ae0a2fb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6399b339d8404cb5da05a7ae0a2fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44445 } } }
I20260806 22:39:50.152436 13267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:50.152592 13269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0 [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:50.153548 13274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:39:50.161156 13274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:39:50.173703 13274 catalog_manager.cc:1347] Generated new cluster ID: 3d483e9007034bd08c0b8b72aee003c9
I20260806 22:39:50.173893 13274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:39:50.186303 13274 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:39:50.186910 13274 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:39:50.197314 13274 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b6399b339d8404cb5da05a7ae0a2fb0: Generated new TSK 0
I20260806 22:39:50.198850 13274 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:39:50.205361 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 13159
W20260806 22:39:50.207121 13245 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43515
W20260806 22:39:50.208154 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 13159: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:39:50.208339 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 13159
2026-08-06T22:39:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2534 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:39:50 dist-test-slave-h3ft krb5kdc[13298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:39:50 dist-test-slave-h3ft krb5kdc[13298](info): set up 2 sockets
Aug 06 22:39:50 dist-test-slave-h3ft krb5kdc[13298](info): commencing operation
krb5kdc: starting...
W20260806 22:39:51.604359 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.340s user 0.000s sys 0.067s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:39:51 dist-test-slave-h3ft krb5kdc[13298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:39:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:39:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:39:52.052595 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42511
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:52.508656 13315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:52.509115 13315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:52.509310 13315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:52.517750 13315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:52.517930 13315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:52.518045 13315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:52.518141 13315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:52.518250 13315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:52.530331 13315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:52.532394 13315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:52.534248 13315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:52.550004 13322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:52.549968 13325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:52.550235 13321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:52.551724 13315 server_base.cc:1034] running on GCE node
I20260806 22:39:52.553012 13315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:52.555287 13315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:52.556602 13315 hybrid_clock.cc:648] HybridClock initialized: now 1786055992556558 us; error 68 us; skew 500 ppm
I20260806 22:39:52.557170 13315 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:52 dist-test-slave-h3ft krb5kdc[13298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055992, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:52.568387 13315 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:39:52.570915 13315 webserver.cc:460] Webserver started at http://127.0.0.1:34833/ using document root <none> and password file <none>
I20260806 22:39:52.572026 13315 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:52.572224 13315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:52.572671 13315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:52.576977 13315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "23b30d2d466e4433837bde2bbaff3fe9"
format_stamp: "Formatted at 2026-08-06 22:39:52 on dist-test-slave-h3ft"
server_key: "dff60a24f50f24f1d27c837ac9f93108"
server_key_iv: "06360dcf7139e08b4fca40aed3686f76"
server_key_version: "encryptionkey@0"
I20260806 22:39:52.578058 13315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "23b30d2d466e4433837bde2bbaff3fe9"
format_stamp: "Formatted at 2026-08-06 22:39:52 on dist-test-slave-h3ft"
server_key: "dff60a24f50f24f1d27c837ac9f93108"
server_key_iv: "06360dcf7139e08b4fca40aed3686f76"
server_key_version: "encryptionkey@0"
I20260806 22:39:52.585414 13315 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 22:39:52.590313 13337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:52.592550 13315 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:39:52.592904 13315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "23b30d2d466e4433837bde2bbaff3fe9"
format_stamp: "Formatted at 2026-08-06 22:39:52 on dist-test-slave-h3ft"
server_key: "dff60a24f50f24f1d27c837ac9f93108"
server_key_iv: "06360dcf7139e08b4fca40aed3686f76"
server_key_version: "encryptionkey@0"
I20260806 22:39:52.593261 13315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:52.621089 13315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:52.627504 13315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:52.627919 13315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:52.653334 13315 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42511
I20260806 22:39:52.653350 13409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42511 every 8 connection(s)
I20260806 22:39:52.655357 13315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:39:52.660935 13410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:52.662798 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 13315
I20260806 22:39:52.663118 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:39:52.663754 16154 external_mini_cluster.cc:1428] Setting key f5dc200edf250edbf856a950e3d31b22
I20260806 22:39:52.681083 13410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9: Bootstrap starting.
Aug 06 22:39:52 dist-test-slave-h3ft krb5kdc[13298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:39:52.686565 13410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:52.688890 13410 log.cc:824] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:52.693744 13410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9: No bootstrap required, opened a new log
I20260806 22:39:52.697213 13413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:52.666493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46419 (local address 127.0.0.1:42511)
0806 22:39:52.667693 (+ 1200us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:52.667719 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:52.667801 (+ 82us) server_negotiation.cc:407] Connection header received
0806 22:39:52.670202 (+ 2401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:52.670251 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:52.671166 (+ 915us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:52.671818 (+ 652us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:52.673804 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:52.676347 (+ 2543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:52.678040 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:52.679510 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:52.679773 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:52.685370 (+ 5597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:52.685387 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:52.685418 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:52.685493 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:52.690802 (+ 5309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:52.691969 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:52.691982 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:52.691996 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:52.692280 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:52.692836 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:52.692844 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:52.692851 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:52.693802 (+ 951us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:52.694173 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:52.694723 (+ 550us) server_negotiation.cc:299] Negotiation successful
0806 22:39:52.695383 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":850,"thread_start_us":517,"threads_started":1}
I20260806 22:39:52.702847 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:39:52.705781 13410 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b30d2d466e4433837bde2bbaff3fe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42511 } }
I20260806 22:39:52.706480 13410 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:52.706673 13410 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b30d2d466e4433837bde2bbaff3fe9, State: Initialized, Role: FOLLOWER
I20260806 22:39:52.707605 13410 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b30d2d466e4433837bde2bbaff3fe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42511 } }
I20260806 22:39:52.707933 13410 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:39:52.708139 13410 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:39:52.708402 13410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:52.712049 13410 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b30d2d466e4433837bde2bbaff3fe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42511 } }
I20260806 22:39:52.712961 13410 leader_election.cc:302] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23b30d2d466e4433837bde2bbaff3fe9; no voters:
I20260806 22:39:52.713888 13410 leader_election.cc:288] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:39:52.714108 13424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:52.715677 13424 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [term 1 LEADER]: Becoming Leader. State: Replica: 23b30d2d466e4433837bde2bbaff3fe9, State: Running, Role: LEADER
I20260806 22:39:52.716586 13424 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b30d2d466e4433837bde2bbaff3fe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42511 } }
I20260806 22:39:52.717356 13410 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:52.720167 13426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23b30d2d466e4433837bde2bbaff3fe9. Latest consensus state: current_term: 1 leader_uuid: "23b30d2d466e4433837bde2bbaff3fe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b30d2d466e4433837bde2bbaff3fe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42511 } } }
I20260806 22:39:52.720671 13426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:52.721450 13425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23b30d2d466e4433837bde2bbaff3fe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b30d2d466e4433837bde2bbaff3fe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42511 } } }
I20260806 22:39:52.721812 13425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:52.724473 13430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:39:52.728929 13413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:52.708457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55625 (local address 127.0.0.1:42511)
0806 22:39:52.708697 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:52.708706 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:52.709351 (+ 645us) server_negotiation.cc:407] Connection header received
0806 22:39:52.709601 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:52.709608 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:52.709722 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:52.709867 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:52.711706 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:52.712926 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:52.714551 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:52.715376 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:52.715563 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:52.719022 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:52.719033 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:52.719040 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:52.719094 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:52.724041 (+ 4947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:52.725135 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:52.725145 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:52.725148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:52.725334 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:52.725992 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:52.726001 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:52.726007 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:52.726691 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:52.727039 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:52.727948 (+ 909us) server_negotiation.cc:299] Negotiation successful
0806 22:39:52.728335 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 22:39:52.737354 13430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:39:52.753539 13430 catalog_manager.cc:1347] Generated new cluster ID: d2f9306c976f4e21abfd01bef73f9add
I20260806 22:39:52.753834 13430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:39:52.767333 13430 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:39:52.769528 13430 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:39:52.783846 13430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23b30d2d466e4433837bde2bbaff3fe9: Generated new TSK 0
I20260806 22:39:52.785494 13430 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:39:52.793476 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 13315
I20260806 22:39:52.795077 13401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:39:52.907809 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 13315
2026-08-06T22:39:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2699 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:39:53 dist-test-slave-h3ft krb5kdc[13451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:39:53 dist-test-slave-h3ft krb5kdc[13451](info): set up 2 sockets
Aug 06 22:39:53 dist-test-slave-h3ft krb5kdc[13451](info): commencing operation
krb5kdc: starting...
W20260806 22:39:54.380360 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.416s user 0.003s sys 0.066s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:39:54 dist-test-slave-h3ft krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:39:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:39:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:39:54.835760 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38085
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:55.317998 13468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:55.318437 13468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:55.318547 13468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:55.326886 13468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:55.327140 13468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:55.327219 13468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:55.327293 13468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:55.327335 13468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:55.338987 13468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:55.340850 13468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:55.342497 13468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:55.356937 13473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:55.357013 13474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:55.356961 13477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:55.358546 13468 server_base.cc:1034] running on GCE node
I20260806 22:39:55.359874 13468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:55.361904 13468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:55.363279 13468 hybrid_clock.cc:648] HybridClock initialized: now 1786055995363232 us; error 40 us; skew 500 ppm
I20260806 22:39:55.363761 13468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:55 dist-test-slave-h3ft krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055995, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:55.367884 13468 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:39:55.369884 13468 webserver.cc:460] Webserver started at http://127.0.0.1:46883/ using document root <none> and password file <none>
I20260806 22:39:55.370760 13468 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:55.370879 13468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:55.371302 13468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:55.375229 13468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "d97cdbfc6105444abc198ab1247032bd"
format_stamp: "Formatted at 2026-08-06 22:39:55 on dist-test-slave-h3ft"
server_key: "3ed6eea147e35ad11cb9cf98a853c37e"
server_key_iv: "249e441b098156ded12906382cb64aef"
server_key_version: "encryptionkey@0"
I20260806 22:39:55.376145 13468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "d97cdbfc6105444abc198ab1247032bd"
format_stamp: "Formatted at 2026-08-06 22:39:55 on dist-test-slave-h3ft"
server_key: "3ed6eea147e35ad11cb9cf98a853c37e"
server_key_iv: "249e441b098156ded12906382cb64aef"
server_key_version: "encryptionkey@0"
I20260806 22:39:55.382928 13468 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260806 22:39:55.387841 13490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:55.389933 13468 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 22:39:55.390211 13468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "d97cdbfc6105444abc198ab1247032bd"
format_stamp: "Formatted at 2026-08-06 22:39:55 on dist-test-slave-h3ft"
server_key: "3ed6eea147e35ad11cb9cf98a853c37e"
server_key_iv: "249e441b098156ded12906382cb64aef"
server_key_version: "encryptionkey@0"
I20260806 22:39:55.390472 13468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:55.421238 13468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:55.426965 13468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:55.427438 13468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:55.451998 13468 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38085
I20260806 22:39:55.452028 13558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38085 every 8 connection(s)
I20260806 22:39:55.454013 13468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:39:55.458926 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 13468
I20260806 22:39:55.459227 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:39:55.459766 13560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:55.459923 16154 external_mini_cluster.cc:1428] Setting key 14fcc48b6dc970fb3693e5b28279e954
I20260806 22:39:55.478375 13563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:55.462428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60849 (local address 127.0.0.1:38085)
0806 22:39:55.463616 (+ 1188us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:55.463632 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:55.463701 (+ 69us) server_negotiation.cc:407] Connection header received
0806 22:39:55.466028 (+ 2327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:55.466061 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:55.466635 (+ 574us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:55.467197 (+ 562us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:55.468031 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.469777 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.472094 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.473077 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.473257 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:55.475113 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:55.475131 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:55.475158 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:55.475967 (+ 809us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:55.476089 (+ 122us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:55.476124 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:55.476239 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:55.476530 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 22:39:55.476987 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":903,"thread_start_us":572,"threads_started":1}
I20260806 22:39:55.478816 13560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd: Bootstrap starting.
I20260806 22:39:55.483882 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:39:55.483984 13560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:55.485452 13560 log.cc:824] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:55.490011 13560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd: No bootstrap required, opened a new log
I20260806 22:39:55.498975 13563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:55.490144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41469 (local address 127.0.0.1:38085)
0806 22:39:55.490423 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:55.490431 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:55.490930 (+ 499us) server_negotiation.cc:407] Connection header received
0806 22:39:55.491278 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:55.491289 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:55.491383 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:55.491486 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:55.492369 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.493309 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.495726 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.496322 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.496514 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:55.498029 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:55.498035 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:55.498039 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:55.498066 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:55.498123 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:55.498146 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:55.498228 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:55.498413 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 22:39:55.498534 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 22:39:55.502731 13560 raft_consensus.cc:348] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97cdbfc6105444abc198ab1247032bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38085 } }
I20260806 22:39:55.503418 13560 raft_consensus.cc:374] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:55.503561 13560 raft_consensus.cc:729] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d97cdbfc6105444abc198ab1247032bd, State: Initialized, Role: FOLLOWER
I20260806 22:39:55.504400 13560 consensus_queue.cc:260] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97cdbfc6105444abc198ab1247032bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38085 } }
I20260806 22:39:55.504747 13560 raft_consensus.cc:388] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:39:55.504938 13560 raft_consensus.cc:482] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:39:55.505168 13560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:55.508315 13560 raft_consensus.cc:504] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97cdbfc6105444abc198ab1247032bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38085 } }
I20260806 22:39:55.509048 13560 leader_election.cc:302] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d97cdbfc6105444abc198ab1247032bd; no voters:
I20260806 22:39:55.509832 13560 leader_election.cc:288] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:39:55.509996 13572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:55.511577 13572 raft_consensus.cc:686] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [term 1 LEADER]: Becoming Leader. State: Replica: d97cdbfc6105444abc198ab1247032bd, State: Running, Role: LEADER
I20260806 22:39:55.512251 13572 consensus_queue.cc:237] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97cdbfc6105444abc198ab1247032bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38085 } }
I20260806 22:39:55.513000 13560 sys_catalog.cc:564] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:55.516242 13576 sys_catalog.cc:455] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader d97cdbfc6105444abc198ab1247032bd. Latest consensus state: current_term: 1 leader_uuid: "d97cdbfc6105444abc198ab1247032bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97cdbfc6105444abc198ab1247032bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38085 } } }
I20260806 22:39:55.516273 13575 sys_catalog.cc:455] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d97cdbfc6105444abc198ab1247032bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97cdbfc6105444abc198ab1247032bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38085 } } }
I20260806 22:39:55.517781 13575 sys_catalog.cc:458] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:55.518291 13576 sys_catalog.cc:458] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:55.520956 13585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:39:55.527096 13585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:39:55.538647 13585 catalog_manager.cc:1347] Generated new cluster ID: 94bc91eca0134de9b8e2605137654de4
I20260806 22:39:55.538830 13585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:39:55.555037 13585 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:39:55.557612 13585 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:39:55.569222 13585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d97cdbfc6105444abc198ab1247032bd: Generated new TSK 0
I20260806 22:39:55.570778 13585 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:39:55.580353 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 13468
W20260806 22:39:55.581979 13554 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60849
W20260806 22:39:55.583083 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 13468: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:39:55.583267 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 13468
2026-08-06T22:39:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2676 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:39:55 dist-test-slave-h3ft krb5kdc[13603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:39:55 dist-test-slave-h3ft krb5kdc[13603](info): set up 2 sockets
Aug 06 22:39:55 dist-test-slave-h3ft krb5kdc[13603](info): commencing operation
krb5kdc: starting...
W20260806 22:39:56.995373 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.355s user 0.002s sys 0.072s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:39:57 dist-test-slave-h3ft krb5kdc[13603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:39:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:39:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:39:57.452395 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34295
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:57.905164 13618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:57.905649 13618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:57.905841 13618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:57.914275 13618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:57.914508 13618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:57.914630 13618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:57.914754 13618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:57.914846 13618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:57.926677 13618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:39:57.928658 13618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:57.930539 13618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:57.946493 13623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:57.946365 13626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:57.946447 13624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:57.947974 13618 server_base.cc:1034] running on GCE node
I20260806 22:39:57.949050 13618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:57.951292 13618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:57.952551 13618 hybrid_clock.cc:648] HybridClock initialized: now 1786055997952462 us; error 86 us; skew 500 ppm
I20260806 22:39:57.952972 13618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:39:57 dist-test-slave-h3ft krb5kdc[13603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055997, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:57.964102 13618 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:39:57.966526 13618 webserver.cc:460] Webserver started at http://127.0.0.1:43133/ using document root <none> and password file <none>
I20260806 22:39:57.967670 13618 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:57.967849 13618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:57.968305 13618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:57.972818 13618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "0b45196338ac487da12c53d8e3980f2a"
format_stamp: "Formatted at 2026-08-06 22:39:57 on dist-test-slave-h3ft"
server_key: "284d5b3a6559350279bff84c18944198"
server_key_iv: "e7109d562183d5820420e3322c794014"
server_key_version: "encryptionkey@0"
I20260806 22:39:57.973866 13618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "0b45196338ac487da12c53d8e3980f2a"
format_stamp: "Formatted at 2026-08-06 22:39:57 on dist-test-slave-h3ft"
server_key: "284d5b3a6559350279bff84c18944198"
server_key_iv: "e7109d562183d5820420e3322c794014"
server_key_version: "encryptionkey@0"
I20260806 22:39:57.981196 13618 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 22:39:57.985858 13638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:57.987943 13618 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:39:57.988202 13618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "0b45196338ac487da12c53d8e3980f2a"
format_stamp: "Formatted at 2026-08-06 22:39:57 on dist-test-slave-h3ft"
server_key: "284d5b3a6559350279bff84c18944198"
server_key_iv: "e7109d562183d5820420e3322c794014"
server_key_version: "encryptionkey@0"
I20260806 22:39:57.988474 13618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:57.996222 13618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:58.001942 13618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:58.002326 13618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:58.025974 13618 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34295
I20260806 22:39:58.026096 13707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34295 every 8 connection(s)
I20260806 22:39:58.027874 13618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:39:58.032583 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 13618
I20260806 22:39:58.032864 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:39:58.033149 13708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:58.033499 16154 external_mini_cluster.cc:1428] Setting key 026771104f731f285395d26632be6bb2
I20260806 22:39:58.044003 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.035832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56619 (local address 127.0.0.1:34295)
0806 22:39:58.036978 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.036997 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.037060 (+ 63us) server_negotiation.cc:407] Connection header received
0806 22:39:58.039181 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.039214 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.039742 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.040179 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.041677 (+ 1498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.041686 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.041703 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:58.041726 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.041822 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:58.041887 (+ 65us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.042157 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.042300 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.042650 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":842,"thread_start_us":510,"threads_started":1}
I20260806 22:39:58.049580 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:39:58.050964 13708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a: Bootstrap starting.
I20260806 22:39:58.056424 13708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:58.058282 13708 log.cc:824] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:58.060030 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.056654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49257 (local address 127.0.0.1:34295)
0806 22:39:58.056966 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.056976 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.057349 (+ 373us) server_negotiation.cc:407] Connection header received
0806 22:39:58.057562 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.057567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.057633 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.057769 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.059058 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.059067 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.059074 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:39:58.059118 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.059186 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:39:58.059218 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.059380 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.059560 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.059637 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260806 22:39:58.062592 13708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a: No bootstrap required, opened a new log
I20260806 22:39:58.071404 13708 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b45196338ac487da12c53d8e3980f2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34295 } }
I20260806 22:39:58.071906 13708 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:58.072013 13708 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b45196338ac487da12c53d8e3980f2a, State: Initialized, Role: FOLLOWER
I20260806 22:39:58.072897 13708 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b45196338ac487da12c53d8e3980f2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34295 } }
I20260806 22:39:58.073267 13708 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:39:58.073462 13708 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:39:58.073660 13708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:58.076969 13708 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b45196338ac487da12c53d8e3980f2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34295 } }
I20260806 22:39:58.077674 13708 leader_election.cc:302] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b45196338ac487da12c53d8e3980f2a; no voters:
I20260806 22:39:58.078470 13708 leader_election.cc:288] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:39:58.078720 13724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:58.079684 13724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [term 1 LEADER]: Becoming Leader. State: Replica: 0b45196338ac487da12c53d8e3980f2a, State: Running, Role: LEADER
I20260806 22:39:58.080536 13724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b45196338ac487da12c53d8e3980f2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34295 } }
I20260806 22:39:58.081560 13708 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:58.084585 13725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b45196338ac487da12c53d8e3980f2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b45196338ac487da12c53d8e3980f2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34295 } } }
I20260806 22:39:58.085099 13725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:58.085664 13726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b45196338ac487da12c53d8e3980f2a. Latest consensus state: current_term: 1 leader_uuid: "0b45196338ac487da12c53d8e3980f2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b45196338ac487da12c53d8e3980f2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34295 } } }
I20260806 22:39:58.086030 13726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:58.088603 13733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:39:58.095628 13733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:39:58.108327 13733 catalog_manager.cc:1347] Generated new cluster ID: efe5a5db482541ae8e9e516d24bb1469
I20260806 22:39:58.108551 13733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:39:58.173882 13733 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:39:58.174499 13733 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:39:58.183843 13733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b45196338ac487da12c53d8e3980f2a: Generated new TSK 0
I20260806 22:39:58.185150 13733 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:39:58.196661 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 13618
W20260806 22:39:58.198217 13703 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56619
W20260806 22:39:58.199281 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 13618: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:39:58.199478 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 13618
2026-08-06T22:39:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2623 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:39:58 dist-test-slave-h3ft krb5kdc[13749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:39:58 dist-test-slave-h3ft krb5kdc[13749](info): set up 2 sockets
Aug 06 22:39:58 dist-test-slave-h3ft krb5kdc[13749](info): commencing operation
krb5kdc: starting...
W20260806 22:39:59.671458 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.405s user 0.002s sys 0.068s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:39:59 dist-test-slave-h3ft krb5kdc[13749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:39:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:39:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:40:00.128321 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45585
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:00.574023 13765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:00.574468 13765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:00.574668 13765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:00.583189 13765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:00.583369 13765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:00.583463 13765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:00.583588 13765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:00.583664 13765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:00.595748 13765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:00.597661 13765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:00.599583 13765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:00.615293 13775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:00.615252 13776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:00.615348 13779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:00.616441 13765 server_base.cc:1034] running on GCE node
I20260806 22:40:00.617537 13765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:00.619558 13765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:00.620802 13765 hybrid_clock.cc:648] HybridClock initialized: now 1786056000620742 us; error 46 us; skew 500 ppm
I20260806 22:40:00.621330 13765 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:00 dist-test-slave-h3ft krb5kdc[13749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056000, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:00.625792 13765 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:40:00.627933 13765 webserver.cc:460] Webserver started at http://127.0.0.1:34737/ using document root <none> and password file <none>
I20260806 22:40:00.628827 13765 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:00.628950 13765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:00.629276 13765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:00.633170 13765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "eff332e400d4490494c9ea044df581cd"
format_stamp: "Formatted at 2026-08-06 22:40:00 on dist-test-slave-h3ft"
server_key: "64534638366df366fa927249c80c9999"
server_key_iv: "412eeb0ec5b8a46fc792b79210f5fd91"
server_key_version: "encryptionkey@0"
I20260806 22:40:00.634095 13765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "eff332e400d4490494c9ea044df581cd"
format_stamp: "Formatted at 2026-08-06 22:40:00 on dist-test-slave-h3ft"
server_key: "64534638366df366fa927249c80c9999"
server_key_iv: "412eeb0ec5b8a46fc792b79210f5fd91"
server_key_version: "encryptionkey@0"
I20260806 22:40:00.640745 13765 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.007s
I20260806 22:40:00.645375 13788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:00.647321 13765 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:40:00.647568 13765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "eff332e400d4490494c9ea044df581cd"
format_stamp: "Formatted at 2026-08-06 22:40:00 on dist-test-slave-h3ft"
server_key: "64534638366df366fa927249c80c9999"
server_key_iv: "412eeb0ec5b8a46fc792b79210f5fd91"
server_key_version: "encryptionkey@0"
I20260806 22:40:00.647838 13765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:00.673919 13765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:00.680408 13765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:00.680837 13765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:00.706502 13765 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45585
I20260806 22:40:00.706517 13867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45585 every 8 connection(s)
I20260806 22:40:00.708549 13765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:00.713811 13871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:00.718194 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 13765
I20260806 22:40:00.718449 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:40:00.719053 16154 external_mini_cluster.cc:1428] Setting key 4e796c121c47d94cd0b85863e226b3b3
I20260806 22:40:00.730496 13871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd: Bootstrap starting.
I20260806 22:40:00.735459 13871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:00.736857 13871 log.cc:824] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd: Log is configured to *not* fsync() on all Append() calls
Aug 06 22:40:00 dist-test-slave-h3ft krb5kdc[13749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:40:00.741451 13871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd: No bootstrap required, opened a new log
I20260806 22:40:00.750581 13874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:00.721594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:44907 (local address 127.0.0.1:45585)
0806 22:40:00.722913 (+ 1319us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:00.722932 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:00.723027 (+ 95us) server_negotiation.cc:407] Connection header received
0806 22:40:00.725332 (+ 2305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:00.725378 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:00.726149 (+ 771us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:00.726741 (+ 592us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:00.728576 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:00.730759 (+ 2183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:00.732673 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:00.733942 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:00.734186 (+ 244us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:00.739483 (+ 5297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:00.739495 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:00.739521 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:00.739582 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:00.744320 (+ 4738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:00.745415 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:00.745431 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:00.745448 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:00.745732 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:00.746328 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:00.746334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:00.746338 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:00.747262 (+ 924us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:00.747685 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:00.748248 (+ 563us) server_negotiation.cc:299] Negotiation successful
0806 22:40:00.748730 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":998,"thread_start_us":574,"threads_started":1}
I20260806 22:40:00.753402 13871 raft_consensus.cc:348] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff332e400d4490494c9ea044df581cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45585 } }
I20260806 22:40:00.753906 13871 raft_consensus.cc:374] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:00.754052 13871 raft_consensus.cc:729] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eff332e400d4490494c9ea044df581cd, State: Initialized, Role: FOLLOWER
I20260806 22:40:00.754866 13871 consensus_queue.cc:260] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff332e400d4490494c9ea044df581cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45585 } }
I20260806 22:40:00.755241 13871 raft_consensus.cc:388] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:00.755416 13871 raft_consensus.cc:482] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:00.755641 13871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:00.756124 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:00.758317 13871 raft_consensus.cc:504] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff332e400d4490494c9ea044df581cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45585 } }
I20260806 22:40:00.759059 13871 leader_election.cc:302] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eff332e400d4490494c9ea044df581cd; no voters:
I20260806 22:40:00.759912 13871 leader_election.cc:288] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:00.760041 13881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:00.761446 13881 raft_consensus.cc:686] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [term 1 LEADER]: Becoming Leader. State: Replica: eff332e400d4490494c9ea044df581cd, State: Running, Role: LEADER
I20260806 22:40:00.762498 13881 consensus_queue.cc:237] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff332e400d4490494c9ea044df581cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45585 } }
I20260806 22:40:00.763147 13871 sys_catalog.cc:564] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:00.766383 13887 sys_catalog.cc:455] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eff332e400d4490494c9ea044df581cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff332e400d4490494c9ea044df581cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45585 } } }
I20260806 22:40:00.766939 13887 sys_catalog.cc:458] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:00.767792 13888 sys_catalog.cc:455] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader eff332e400d4490494c9ea044df581cd. Latest consensus state: current_term: 1 leader_uuid: "eff332e400d4490494c9ea044df581cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff332e400d4490494c9ea044df581cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45585 } } }
I20260806 22:40:00.768214 13888 sys_catalog.cc:458] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:00.780616 13893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:00.785171 13874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:00.761997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:42565 (local address 127.0.0.1:45585)
0806 22:40:00.762207 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:00.762214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:00.762969 (+ 755us) server_negotiation.cc:407] Connection header received
0806 22:40:00.763270 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:00.763278 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:00.763383 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:00.763500 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:00.765798 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:00.767154 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:00.769955 (+ 2801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:00.771043 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:00.771199 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:00.776089 (+ 4890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:00.776096 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:00.776100 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:00.776148 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:00.780392 (+ 4244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:00.781658 (+ 1266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:00.781667 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:00.781674 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:00.781900 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:00.782571 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:00.782580 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:00.782586 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:00.783222 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:00.783598 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:00.784268 (+ 670us) server_negotiation.cc:299] Negotiation successful
0806 22:40:00.784567 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:40:00.791018 13893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:00.803566 13893 catalog_manager.cc:1347] Generated new cluster ID: 0591e2610dc748a8a95750770d2eec06
I20260806 22:40:00.803709 13893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:00.816824 13893 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:00.819197 13893 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:00.831715 13893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eff332e400d4490494c9ea044df581cd: Generated new TSK 0
I20260806 22:40:00.832955 13893 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:00.841426 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 13765
I20260806 22:40:00.844442 13861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:40:00.954929 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 13765
W20260806 22:40:00.988610 13746 connection.cc:535] client connection to 127.0.0.1:45585 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45585): Connection reset by peer (error 104)
2026-08-06T22:40:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2747 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:01 dist-test-slave-h3ft krb5kdc[13918](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:01 dist-test-slave-h3ft krb5kdc[13918](info): set up 2 sockets
Aug 06 22:40:01 dist-test-slave-h3ft krb5kdc[13918](info): commencing operation
krb5kdc: starting...
W20260806 22:40:02.397172 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.386s user 0.006s sys 0.063s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:40:02 dist-test-slave-h3ft krb5kdc[13918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
W20260806 22:40:02.880218 16154 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.0.0.1: real 0.101s user 0.003s sys 0.034s
I20260806 22:40:02.881454 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40775
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:03.370517 13934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:03.370952 13934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:03.371172 13934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:03.379881 13934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:03.380074 13934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:03.380136 13934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:03.380232 13934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:03.380313 13934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:03.392621 13934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:03.394531 13934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:03.396322 13934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:03.412899 13939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:03.412994 13940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:03.412940 13944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:03.417629 13934 server_base.cc:1034] running on GCE node
I20260806 22:40:03.418879 13934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:03.421159 13934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:03.422466 13934 hybrid_clock.cc:648] HybridClock initialized: now 1786056003422392 us; error 58 us; skew 500 ppm
I20260806 22:40:03.423125 13934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:03 dist-test-slave-h3ft krb5kdc[13918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:03.431303 13934 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:40:03.433575 13934 webserver.cc:460] Webserver started at http://127.0.0.1:33583/ using document root <none> and password file <none>
I20260806 22:40:03.434587 13934 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:03.434762 13934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:03.435257 13934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:03.439538 13934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "24845e30160e4bb391b5f00f1a4d443f"
format_stamp: "Formatted at 2026-08-06 22:40:03 on dist-test-slave-h3ft"
server_key: "21249c1c5164266cc10ca5416a14c190"
server_key_iv: "be2c4673bde1234b4ee984d961d3384c"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.440688 13934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "24845e30160e4bb391b5f00f1a4d443f"
format_stamp: "Formatted at 2026-08-06 22:40:03 on dist-test-slave-h3ft"
server_key: "21249c1c5164266cc10ca5416a14c190"
server_key_iv: "be2c4673bde1234b4ee984d961d3384c"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.447912 13934 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260806 22:40:03.452844 13954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.454823 13934 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260806 22:40:03.455139 13934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "24845e30160e4bb391b5f00f1a4d443f"
format_stamp: "Formatted at 2026-08-06 22:40:03 on dist-test-slave-h3ft"
server_key: "21249c1c5164266cc10ca5416a14c190"
server_key_iv: "be2c4673bde1234b4ee984d961d3384c"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.455396 13934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:03.489228 13934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:03.496019 13934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:03.496464 13934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:03.521248 13934 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40775
I20260806 22:40:03.521291 14036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40775 every 8 connection(s)
I20260806 22:40:03.523272 13934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:03.524538 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 13934
I20260806 22:40:03.524816 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:40:03.525586 16154 external_mini_cluster.cc:1428] Setting key 0b0eb6367b4e0c46eb268f6b403eebba
I20260806 22:40:03.529764 14037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:03.545632 14040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.528209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:49729 (local address 127.0.0.1:40775)
0806 22:40:03.529232 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.529248 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.529304 (+ 56us) server_negotiation.cc:407] Connection header received
0806 22:40:03.531709 (+ 2405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.531743 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.532355 (+ 612us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.532912 (+ 557us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:03.534362 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.536801 (+ 2439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.538505 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.539846 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.540086 (+ 240us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.541698 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:03.541709 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:03.541736 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:03.542733 (+ 997us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:03.542835 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:03.542889 (+ 54us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:03.543122 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.543568 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.543830 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":756,"thread_start_us":455,"threads_started":1}
I20260806 22:40:03.548041 14037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f: Bootstrap starting.
I20260806 22:40:03.551225 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:03.552824 14037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:03.554237 14037 log.cc:824] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:03.558897 14037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f: No bootstrap required, opened a new log
I20260806 22:40:03.566207 14040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.557428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:50521 (local address 127.0.0.1:40775)
0806 22:40:03.557718 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.557728 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.558228 (+ 500us) server_negotiation.cc:407] Connection header received
0806 22:40:03.558423 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.558429 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.558525 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.558655 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:03.559629 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.560900 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.562683 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.563507 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.563671 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.565196 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:03.565208 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:03.565214 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:03.565244 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:03.565332 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:03.565363 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:03.565490 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.565712 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.565787 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 22:40:03.569808 14037 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24845e30160e4bb391b5f00f1a4d443f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } }
I20260806 22:40:03.570293 14037 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:03.570390 14037 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24845e30160e4bb391b5f00f1a4d443f, State: Initialized, Role: FOLLOWER
I20260806 22:40:03.571208 14037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24845e30160e4bb391b5f00f1a4d443f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } }
I20260806 22:40:03.571484 14037 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:03.571616 14037 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:03.571770 14037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:03.574414 14037 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24845e30160e4bb391b5f00f1a4d443f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } }
I20260806 22:40:03.575232 14037 leader_election.cc:302] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24845e30160e4bb391b5f00f1a4d443f; no voters:
I20260806 22:40:03.576010 14037 leader_election.cc:288] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:03.576198 14052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:03.577032 14052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [term 1 LEADER]: Becoming Leader. State: Replica: 24845e30160e4bb391b5f00f1a4d443f, State: Running, Role: LEADER
I20260806 22:40:03.577768 14052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24845e30160e4bb391b5f00f1a4d443f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } }
I20260806 22:40:03.578900 14037 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:03.582170 14054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24845e30160e4bb391b5f00f1a4d443f. Latest consensus state: current_term: 1 leader_uuid: "24845e30160e4bb391b5f00f1a4d443f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24845e30160e4bb391b5f00f1a4d443f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } } }
I20260806 22:40:03.582077 14053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24845e30160e4bb391b5f00f1a4d443f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24845e30160e4bb391b5f00f1a4d443f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } } }
I20260806 22:40:03.582842 14053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:03.582840 14054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:03.585403 14064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:03.592628 14064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:03.606349 14064 catalog_manager.cc:1347] Generated new cluster ID: 98887cb7171948df872d78dc93ee4df8
I20260806 22:40:03.606539 14064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:03.652273 14064 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:03.654239 14064 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:03.662228 14064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24845e30160e4bb391b5f00f1a4d443f: Generated new TSK 0
I20260806 22:40:03.663582 14064 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:03.674520 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 13934
W20260806 22:40:03.676082 14031 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.166:49729
W20260806 22:40:03.677153 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 13934: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:40:03.677343 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 13934
W20260806 22:40:03.715809 13914 connection.cc:535] client connection to 127.0.0.1:40775 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40775): Connection reset by peer (error 104)
2026-08-06T22:40:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2727 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:03 dist-test-slave-h3ft krb5kdc[14079](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:03 dist-test-slave-h3ft krb5kdc[14079](info): set up 2 sockets
Aug 06 22:40:03 dist-test-slave-h3ft krb5kdc[14079](info): commencing operation
krb5kdc: starting...
W20260806 22:40:05.186625 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.447s user 0.000s sys 0.075s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:40:05 dist-test-slave-h3ft krb5kdc[14079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:40:05.656880 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34323
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:06.166755 14101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:06.167232 14101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:06.167435 14101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:06.176059 14101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:06.176247 14101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:06.176357 14101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:06.176463 14101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:06.176555 14101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:06.189378 14101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:06.191484 14101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:06.193442 14101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:06.210228 14114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:06.210166 14113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:06.210294 14116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:06.213099 14101 server_base.cc:1034] running on GCE node
I20260806 22:40:06.214399 14101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:06.216565 14101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:06.217804 14101 hybrid_clock.cc:648] HybridClock initialized: now 1786056006217752 us; error 58 us; skew 500 ppm
I20260806 22:40:06.218268 14101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:06 dist-test-slave-h3ft krb5kdc[14079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056006, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:06.222698 14101 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:40:06.225126 14101 webserver.cc:460] Webserver started at http://127.0.0.1:44289/ using document root <none> and password file <none>
I20260806 22:40:06.226140 14101 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:06.226300 14101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:06.226733 14101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:06.231087 14101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "d31d725920034fdcab8a71847d4c187b"
format_stamp: "Formatted at 2026-08-06 22:40:06 on dist-test-slave-h3ft"
server_key: "bf32a40ec40881bb86ac63d5ca2cf5ec"
server_key_iv: "72b3004efc33b1c1e324016637bb2f26"
server_key_version: "encryptionkey@0"
I20260806 22:40:06.232157 14101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "d31d725920034fdcab8a71847d4c187b"
format_stamp: "Formatted at 2026-08-06 22:40:06 on dist-test-slave-h3ft"
server_key: "bf32a40ec40881bb86ac63d5ca2cf5ec"
server_key_iv: "72b3004efc33b1c1e324016637bb2f26"
server_key_version: "encryptionkey@0"
I20260806 22:40:06.239557 14101 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260806 22:40:06.244612 14124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:06.246599 14101 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260806 22:40:06.246855 14101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "d31d725920034fdcab8a71847d4c187b"
format_stamp: "Formatted at 2026-08-06 22:40:06 on dist-test-slave-h3ft"
server_key: "bf32a40ec40881bb86ac63d5ca2cf5ec"
server_key_iv: "72b3004efc33b1c1e324016637bb2f26"
server_key_version: "encryptionkey@0"
I20260806 22:40:06.247169 14101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:06.254957 14101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:06.260830 14101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:06.261245 14101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:06.285670 14101 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34323
I20260806 22:40:06.285725 14195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34323 every 8 connection(s)
I20260806 22:40:06.287791 14101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:06.289947 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 14101
I20260806 22:40:06.290186 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:40:06.290732 16154 external_mini_cluster.cc:1428] Setting key 95188e24ee22ab91ac8649ffe006dfc6
I20260806 22:40:06.293778 14196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:06.300881 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.293290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:58439 (local address 127.0.0.1:34323)
0806 22:40:06.294302 (+ 1012us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.294317 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.294372 (+ 55us) server_negotiation.cc:407] Connection header received
0806 22:40:06.296421 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.296450 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.296892 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.297327 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:06.298726 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:06.298734 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:06.298748 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:06.298764 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:06.298854 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:06.298876 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:06.299144 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.299274 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.299580 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":457,"threads_started":1}
I20260806 22:40:06.305935 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:06.308921 14196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b: Bootstrap starting.
I20260806 22:40:06.313791 14196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:06.315261 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.311742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:56993 (local address 127.0.0.1:34323)
0806 22:40:06.312027 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.312038 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.312744 (+ 706us) server_negotiation.cc:407] Connection header received
0806 22:40:06.312933 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.312939 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.313019 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.313156 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:06.314433 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:06.314446 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:06.314453 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:06.314476 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:06.314547 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:06.314571 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:06.314710 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.314817 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.314896 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 22:40:06.315659 14196 log.cc:824] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:06.319564 14196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b: No bootstrap required, opened a new log
I20260806 22:40:06.328567 14196 raft_consensus.cc:348] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31d725920034fdcab8a71847d4c187b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } }
I20260806 22:40:06.329231 14196 raft_consensus.cc:374] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:06.329432 14196 raft_consensus.cc:729] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d31d725920034fdcab8a71847d4c187b, State: Initialized, Role: FOLLOWER
I20260806 22:40:06.330343 14196 consensus_queue.cc:260] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31d725920034fdcab8a71847d4c187b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } }
I20260806 22:40:06.330653 14196 raft_consensus.cc:388] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:06.330844 14196 raft_consensus.cc:482] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:06.331118 14196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:06.333810 14196 raft_consensus.cc:504] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31d725920034fdcab8a71847d4c187b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } }
I20260806 22:40:06.334489 14196 leader_election.cc:302] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d31d725920034fdcab8a71847d4c187b; no voters:
I20260806 22:40:06.335357 14196 leader_election.cc:288] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:06.335578 14215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:06.337159 14215 raft_consensus.cc:686] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [term 1 LEADER]: Becoming Leader. State: Replica: d31d725920034fdcab8a71847d4c187b, State: Running, Role: LEADER
I20260806 22:40:06.337867 14215 consensus_queue.cc:237] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31d725920034fdcab8a71847d4c187b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } }
I20260806 22:40:06.338215 14196 sys_catalog.cc:564] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:06.342015 14216 sys_catalog.cc:455] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d31d725920034fdcab8a71847d4c187b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31d725920034fdcab8a71847d4c187b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } } }
I20260806 22:40:06.342620 14216 sys_catalog.cc:458] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:06.342006 14217 sys_catalog.cc:455] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d31d725920034fdcab8a71847d4c187b. Latest consensus state: current_term: 1 leader_uuid: "d31d725920034fdcab8a71847d4c187b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31d725920034fdcab8a71847d4c187b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } } }
I20260806 22:40:06.343466 14217 sys_catalog.cc:458] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:06.344929 14228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:06.351150 14228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:06.363821 14228 catalog_manager.cc:1347] Generated new cluster ID: e82526738fe7451599ff5ae0f0cc064e
I20260806 22:40:06.364014 14228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:06.382561 14228 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:06.383303 14228 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:06.391728 14228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d31d725920034fdcab8a71847d4c187b: Generated new TSK 0
I20260806 22:40:06.393118 14228 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:06.416109 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 14101
W20260806 22:40:06.417778 14188 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.166:58439
W20260806 22:40:06.418913 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 14101: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:40:06.419200 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 14101
2026-08-06T22:40:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2742 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:06 dist-test-slave-h3ft krb5kdc[14243](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:06 dist-test-slave-h3ft krb5kdc[14243](info): set up 2 sockets
Aug 06 22:40:06 dist-test-slave-h3ft krb5kdc[14243](info): commencing operation
krb5kdc: starting...
W20260806 22:40:07.941480 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.460s user 0.000s sys 0.072s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:40:08 dist-test-slave-h3ft krb5kdc[14243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:40:08.420512 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35633
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:08.892061 14262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:08.892526 14262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:08.892732 14262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:08.901398 14262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:08.901573 14262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:08.901916 14262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:08.902083 14262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:08.902182 14262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:08.914228 14262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35633
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:08.916175 14262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:08.917943 14262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:08.933328 14268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:08.933255 14269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:08.933231 14271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:08.934629 14262 server_base.cc:1034] running on GCE node
I20260806 22:40:08.935976 14262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:08.938261 14262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:08.939553 14262 hybrid_clock.cc:648] HybridClock initialized: now 1786056008939496 us; error 49 us; skew 500 ppm
I20260806 22:40:08.940096 14262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:08 dist-test-slave-h3ft krb5kdc[14243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056008, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:08.949604 14262 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:40:08.951921 14262 webserver.cc:460] Webserver started at http://127.0.0.1:40783/ using document root <none> and password file <none>
I20260806 22:40:08.952991 14262 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:08.953214 14262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:08.953672 14262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:08.959093 14262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "0de43a90ccd043aab2a6dd4f93a9d1e2"
format_stamp: "Formatted at 2026-08-06 22:40:08 on dist-test-slave-h3ft"
server_key: "b443ee392fee68b198d80113f731b585"
server_key_iv: "83aa154fe11792781cfa1fea86aeabdd"
server_key_version: "encryptionkey@0"
I20260806 22:40:08.960343 14262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "0de43a90ccd043aab2a6dd4f93a9d1e2"
format_stamp: "Formatted at 2026-08-06 22:40:08 on dist-test-slave-h3ft"
server_key: "b443ee392fee68b198d80113f731b585"
server_key_iv: "83aa154fe11792781cfa1fea86aeabdd"
server_key_version: "encryptionkey@0"
I20260806 22:40:08.968745 14262 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260806 22:40:08.974256 14279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:08.976512 14262 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 22:40:08.976806 14262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "0de43a90ccd043aab2a6dd4f93a9d1e2"
format_stamp: "Formatted at 2026-08-06 22:40:08 on dist-test-slave-h3ft"
server_key: "b443ee392fee68b198d80113f731b585"
server_key_iv: "83aa154fe11792781cfa1fea86aeabdd"
server_key_version: "encryptionkey@0"
I20260806 22:40:08.977169 14262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:08.999987 14262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:09.006390 14262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:09.006850 14262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:09.033018 14262 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35633
I20260806 22:40:09.033048 14352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35633 every 8 connection(s)
I20260806 22:40:09.035161 14262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:09.040328 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 14262
I20260806 22:40:09.040613 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:40:09.041046 14353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:09.041335 16154 external_mini_cluster.cc:1428] Setting key 9e69c41305c4429bb2f22b39dd1b9faf
I20260806 22:40:09.058746 14353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2: Bootstrap starting.
Aug 06 22:40:09 dist-test-slave-h3ft krb5kdc[14243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:40:09.063998 14353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:09.065462 14353 log.cc:824] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:09.070168 14353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2: No bootstrap required, opened a new log
I20260806 22:40:09.071668 14358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:09.043916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:44351 (local address 127.0.0.1:35633)
0806 22:40:09.044998 (+ 1082us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:09.045015 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:09.045075 (+ 60us) server_negotiation.cc:407] Connection header received
0806 22:40:09.047674 (+ 2599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:09.047712 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:09.048405 (+ 693us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:09.049040 (+ 635us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:09.051120 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:09.052661 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:09.054230 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:09.055301 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:09.055560 (+ 259us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:09.061411 (+ 5851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:09.061426 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:09.061449 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:09.061512 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:09.065624 (+ 4112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:09.067052 (+ 1428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:09.067064 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:09.067078 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:09.067381 (+ 303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:09.068006 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:09.068010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:09.068012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:09.068687 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:09.069001 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:09.069714 (+ 713us) server_negotiation.cc:299] Negotiation successful
0806 22:40:09.070195 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":476,"threads_started":1}
I20260806 22:40:09.077348 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:09.080562 14353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de43a90ccd043aab2a6dd4f93a9d1e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35633 } }
I20260806 22:40:09.081312 14353 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:09.081471 14353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0de43a90ccd043aab2a6dd4f93a9d1e2, State: Initialized, Role: FOLLOWER
I20260806 22:40:09.082465 14353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de43a90ccd043aab2a6dd4f93a9d1e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35633 } }
I20260806 22:40:09.082940 14353 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:09.083273 14353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:09.083680 14353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:09.087836 14353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de43a90ccd043aab2a6dd4f93a9d1e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35633 } }
I20260806 22:40:09.088773 14353 leader_election.cc:302] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0de43a90ccd043aab2a6dd4f93a9d1e2; no voters:
I20260806 22:40:09.089670 14353 leader_election.cc:288] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:09.089910 14368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:09.091480 14368 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [term 1 LEADER]: Becoming Leader. State: Replica: 0de43a90ccd043aab2a6dd4f93a9d1e2, State: Running, Role: LEADER
I20260806 22:40:09.092408 14368 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de43a90ccd043aab2a6dd4f93a9d1e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35633 } }
I20260806 22:40:09.093006 14353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:09.096594 14369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0de43a90ccd043aab2a6dd4f93a9d1e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de43a90ccd043aab2a6dd4f93a9d1e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35633 } } }
I20260806 22:40:09.097097 14369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:09.097826 14370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0de43a90ccd043aab2a6dd4f93a9d1e2. Latest consensus state: current_term: 1 leader_uuid: "0de43a90ccd043aab2a6dd4f93a9d1e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de43a90ccd043aab2a6dd4f93a9d1e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35633 } } }
I20260806 22:40:09.098253 14370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:09.098879 14374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:09.109043 14358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:09.083524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:51861 (local address 127.0.0.1:35633)
0806 22:40:09.083790 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:09.083800 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:09.084268 (+ 468us) server_negotiation.cc:407] Connection header received
0806 22:40:09.084534 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:09.084540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:09.084643 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:09.084794 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:09.086998 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:09.088026 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:09.090072 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:09.091266 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:09.092334 (+ 1068us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:09.095785 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:09.095793 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:09.095798 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:09.095864 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:09.101612 (+ 5748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:09.103649 (+ 2037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:09.103672 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:09.103683 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:09.103952 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:09.104880 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:09.104889 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:09.104895 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:09.105606 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:09.107734 (+ 2128us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:09.108027 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 22:40:09.108390 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 22:40:09.115847 14374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:09.131390 14374 catalog_manager.cc:1347] Generated new cluster ID: e50e9edf0d164649a62d92b8c3b08df3
I20260806 22:40:09.131600 14374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:09.148806 14374 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:09.150599 14374 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:09.168018 14374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0de43a90ccd043aab2a6dd4f93a9d1e2: Generated new TSK 0
I20260806 22:40:09.169457 14374 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:09.184495 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 14262
I20260806 22:40:09.186223 14347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:40:09.293171 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 14262
W20260806 22:40:09.325695 14241 connection.cc:535] client connection to 127.0.0.1:35633 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35633): Connection reset by peer (error 104)
2026-08-06T22:40:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2866 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:09 dist-test-slave-h3ft krb5kdc[14395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:09 dist-test-slave-h3ft krb5kdc[14395](info): set up 2 sockets
Aug 06 22:40:09 dist-test-slave-h3ft krb5kdc[14395](info): commencing operation
krb5kdc: starting...
W20260806 22:40:10.784436 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.437s user 0.001s sys 0.067s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:40:11 dist-test-slave-h3ft krb5kdc[14395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:40:11.232051 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35631
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:11.661990 14412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:11.662489 14412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:11.662688 14412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:11.670956 14412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:11.671204 14412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:11.671330 14412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:11.671469 14412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:11.671566 14412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:11.683373 14412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35631
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:11.685343 14412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:11.687147 14412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:11.702574 14418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:11.702643 14419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:11.702876 14421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:11.703508 14412 server_base.cc:1034] running on GCE node
I20260806 22:40:11.704679 14412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:11.706746 14412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:11.708033 14412 hybrid_clock.cc:648] HybridClock initialized: now 1786056011707984 us; error 40 us; skew 500 ppm
I20260806 22:40:11.708652 14412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:11 dist-test-slave-h3ft krb5kdc[14395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056011, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:11.712757 14412 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:40:11.714851 14412 webserver.cc:460] Webserver started at http://127.0.0.1:43979/ using document root <none> and password file <none>
I20260806 22:40:11.715867 14412 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:11.716004 14412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:11.716357 14412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:11.720645 14412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "df6f3ebe999743088a845f183443e586"
format_stamp: "Formatted at 2026-08-06 22:40:11 on dist-test-slave-h3ft"
server_key: "228336d0531718de3bc66a5f305484d4"
server_key_iv: "a0c8395d7c2ca33cd1d7fab9ac2721e5"
server_key_version: "encryptionkey@0"
I20260806 22:40:11.721666 14412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "df6f3ebe999743088a845f183443e586"
format_stamp: "Formatted at 2026-08-06 22:40:11 on dist-test-slave-h3ft"
server_key: "228336d0531718de3bc66a5f305484d4"
server_key_iv: "a0c8395d7c2ca33cd1d7fab9ac2721e5"
server_key_version: "encryptionkey@0"
I20260806 22:40:11.728951 14412 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260806 22:40:11.733853 14429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:11.735970 14412 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:40:11.736241 14412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "df6f3ebe999743088a845f183443e586"
format_stamp: "Formatted at 2026-08-06 22:40:11 on dist-test-slave-h3ft"
server_key: "228336d0531718de3bc66a5f305484d4"
server_key_iv: "a0c8395d7c2ca33cd1d7fab9ac2721e5"
server_key_version: "encryptionkey@0"
I20260806 22:40:11.736514 14412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:11.765849 14412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:11.771908 14412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:11.772359 14412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:11.796943 14412 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35631
I20260806 22:40:11.796977 14502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35631 every 8 connection(s)
I20260806 22:40:11.798928 14412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:11.799623 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 14412
I20260806 22:40:11.799877 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:40:11.800711 16154 external_mini_cluster.cc:1428] Setting key 08a91cfa793d32f411ec40751a7eaefe
I20260806 22:40:11.805195 14503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:11.817435 14506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:11.803395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:46733 (local address 127.0.0.1:35631)
0806 22:40:11.804380 (+ 985us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:11.804396 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:11.804452 (+ 56us) server_negotiation.cc:407] Connection header received
0806 22:40:11.806586 (+ 2134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:11.806617 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:11.807216 (+ 599us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:11.807728 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:11.808535 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:11.810107 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:11.811793 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:11.812604 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:11.812863 (+ 259us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:11.814429 (+ 1566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:11.814438 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:11.814458 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:11.815231 (+ 773us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:11.815334 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:11.815367 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:11.815602 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:11.815864 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 22:40:11.816121 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":440,"threads_started":1}
I20260806 22:40:11.819464 14503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586: Bootstrap starting.
I20260806 22:40:11.822783 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:11.824223 14503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:11.825593 14503 log.cc:824] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:11.830067 14503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586: No bootstrap required, opened a new log
I20260806 22:40:11.836649 14506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:11.828395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:41855 (local address 127.0.0.1:35631)
0806 22:40:11.828612 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:11.828621 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:11.829124 (+ 503us) server_negotiation.cc:407] Connection header received
0806 22:40:11.829386 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:11.829391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:11.829464 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:11.829587 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:11.830277 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:11.831591 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:11.833263 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:11.834051 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:11.834194 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:11.835642 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:11.835650 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:11.835655 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:11.835684 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:11.835771 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:11.835794 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:11.835952 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:11.836146 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:40:11.836221 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:40:11.841187 14503 raft_consensus.cc:348] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df6f3ebe999743088a845f183443e586" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35631 } }
I20260806 22:40:11.841775 14503 raft_consensus.cc:374] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:11.841918 14503 raft_consensus.cc:729] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df6f3ebe999743088a845f183443e586, State: Initialized, Role: FOLLOWER
I20260806 22:40:11.842830 14503 consensus_queue.cc:260] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df6f3ebe999743088a845f183443e586" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35631 } }
I20260806 22:40:11.843180 14503 raft_consensus.cc:388] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:11.843355 14503 raft_consensus.cc:482] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:11.843576 14503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:11.846839 14503 raft_consensus.cc:504] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df6f3ebe999743088a845f183443e586" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35631 } }
I20260806 22:40:11.847558 14503 leader_election.cc:302] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df6f3ebe999743088a845f183443e586; no voters:
I20260806 22:40:11.848309 14503 leader_election.cc:288] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:11.848426 14518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:11.849366 14518 raft_consensus.cc:686] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [term 1 LEADER]: Becoming Leader. State: Replica: df6f3ebe999743088a845f183443e586, State: Running, Role: LEADER
I20260806 22:40:11.850031 14518 consensus_queue.cc:237] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df6f3ebe999743088a845f183443e586" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35631 } }
I20260806 22:40:11.850986 14503 sys_catalog.cc:564] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:11.853667 14521 sys_catalog.cc:455] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df6f3ebe999743088a845f183443e586. Latest consensus state: current_term: 1 leader_uuid: "df6f3ebe999743088a845f183443e586" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df6f3ebe999743088a845f183443e586" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35631 } } }
I20260806 22:40:11.854252 14521 sys_catalog.cc:458] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:11.854779 14520 sys_catalog.cc:455] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df6f3ebe999743088a845f183443e586" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df6f3ebe999743088a845f183443e586" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35631 } } }
I20260806 22:40:11.855203 14520 sys_catalog.cc:458] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:11.859754 14526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:11.866888 14526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:11.879176 14526 catalog_manager.cc:1347] Generated new cluster ID: 8c9c3dbfe8ca4ac9973107fb0c7703a0
I20260806 22:40:11.879359 14526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:11.890262 14526 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:11.892621 14526 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:11.906445 14526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df6f3ebe999743088a845f183443e586: Generated new TSK 0
I20260806 22:40:11.907881 14526 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:11.918233 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 14412
W20260806 22:40:11.920493 14498 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.166:46733
W20260806 22:40:11.921626 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 14412: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:40:11.921844 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 14412
W20260806 22:40:11.956321 14393 connection.cc:535] client connection to 127.0.0.1:35631 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35631): Connection reset by peer (error 104)
2026-08-06T22:40:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2631 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:12 dist-test-slave-h3ft krb5kdc[14545](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:12 dist-test-slave-h3ft krb5kdc[14545](info): set up 2 sockets
Aug 06 22:40:12 dist-test-slave-h3ft krb5kdc[14545](info): commencing operation
krb5kdc: starting...
W20260806 22:40:13.345590 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.366s user 0.000s sys 0.072s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:40:13 dist-test-slave-h3ft krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:40:13.788586 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42317
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:14.242489 14562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:14.242900 14562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:14.243062 14562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:14.251302 14562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:14.251469 14562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:14.251521 14562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:14.251592 14562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:14.251641 14562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:14.263239 14562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42317
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:14.265022 14562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:14.266834 14562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:14.281780 14569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:14.281824 14570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:14.281731 14572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:14.282835 14562 server_base.cc:1034] running on GCE node
I20260806 22:40:14.284014 14562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:14.285995 14562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:14.287279 14562 hybrid_clock.cc:648] HybridClock initialized: now 1786056014287189 us; error 88 us; skew 500 ppm
I20260806 22:40:14.287781 14562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:14 dist-test-slave-h3ft krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056014, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:14.295838 14562 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:40:14.297947 14562 webserver.cc:460] Webserver started at http://127.0.0.1:45757/ using document root <none> and password file <none>
I20260806 22:40:14.298918 14562 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:14.299118 14562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:14.299523 14562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:14.303593 14562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "861c2ffe933c412a8409dee12d75c34b"
format_stamp: "Formatted at 2026-08-06 22:40:14 on dist-test-slave-h3ft"
server_key: "20e7dfed57891a6958b36341ddef42f3"
server_key_iv: "a4a107293095a7f38c68efb75dc66fab"
server_key_version: "encryptionkey@0"
I20260806 22:40:14.304586 14562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "861c2ffe933c412a8409dee12d75c34b"
format_stamp: "Formatted at 2026-08-06 22:40:14 on dist-test-slave-h3ft"
server_key: "20e7dfed57891a6958b36341ddef42f3"
server_key_iv: "a4a107293095a7f38c68efb75dc66fab"
server_key_version: "encryptionkey@0"
I20260806 22:40:14.311810 14562 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 22:40:14.316745 14579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:14.318871 14562 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.001s
I20260806 22:40:14.319358 14562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "861c2ffe933c412a8409dee12d75c34b"
format_stamp: "Formatted at 2026-08-06 22:40:14 on dist-test-slave-h3ft"
server_key: "20e7dfed57891a6958b36341ddef42f3"
server_key_iv: "a4a107293095a7f38c68efb75dc66fab"
server_key_version: "encryptionkey@0"
I20260806 22:40:14.319695 14562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:14.327574 14562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:14.333427 14562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:14.333851 14562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:14.358484 14562 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42317
I20260806 22:40:14.358525 14656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42317 every 8 connection(s)
I20260806 22:40:14.360455 14562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:14.365761 14657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:14.367925 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 14562
I20260806 22:40:14.368175 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:40:14.368741 16154 external_mini_cluster.cc:1428] Setting key 0acdf5c77da330437299496bf7c568d9
I20260806 22:40:14.379078 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:14.371140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:37759 (local address 127.0.0.1:42317)
0806 22:40:14.372300 (+ 1160us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:14.372316 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:14.372376 (+ 60us) server_negotiation.cc:407] Connection header received
0806 22:40:14.374449 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:14.374478 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:14.374922 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:14.375435 (+ 513us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:14.376796 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:14.376804 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:14.376820 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:14.376840 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:14.376956 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:14.376995 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:14.377216 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:14.377335 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 22:40:14.377652 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":883,"thread_start_us":547,"threads_started":1}
I20260806 22:40:14.380645 14657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b: Bootstrap starting.
I20260806 22:40:14.383826 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:14.384902 14657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:14.386188 14657 log.cc:824] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:14.390578 14657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b: No bootstrap required, opened a new log
I20260806 22:40:14.392766 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:14.389450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:60051 (local address 127.0.0.1:42317)
0806 22:40:14.389718 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:14.389740 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:14.390195 (+ 455us) server_negotiation.cc:407] Connection header received
0806 22:40:14.390462 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:14.390467 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:14.390533 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:14.390665 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:14.391950 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:14.391957 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:14.391969 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:14.391991 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:14.392039 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:14.392058 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:14.392203 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:14.392311 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 22:40:14.392391 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 22:40:14.400622 14657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861c2ffe933c412a8409dee12d75c34b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42317 } }
I20260806 22:40:14.401168 14657 raft_consensus.cc:374] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:14.401292 14657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 861c2ffe933c412a8409dee12d75c34b, State: Initialized, Role: FOLLOWER
I20260806 22:40:14.402025 14657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861c2ffe933c412a8409dee12d75c34b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42317 } }
I20260806 22:40:14.402323 14657 raft_consensus.cc:388] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:14.402496 14657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:14.402649 14657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:14.406085 14657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861c2ffe933c412a8409dee12d75c34b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42317 } }
I20260806 22:40:14.406739 14657 leader_election.cc:302] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 861c2ffe933c412a8409dee12d75c34b; no voters:
I20260806 22:40:14.407527 14657 leader_election.cc:288] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:14.407712 14675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:14.408505 14675 raft_consensus.cc:686] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [term 1 LEADER]: Becoming Leader. State: Replica: 861c2ffe933c412a8409dee12d75c34b, State: Running, Role: LEADER
I20260806 22:40:14.409142 14675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861c2ffe933c412a8409dee12d75c34b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42317 } }
I20260806 22:40:14.410259 14657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:14.412905 14678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 861c2ffe933c412a8409dee12d75c34b. Latest consensus state: current_term: 1 leader_uuid: "861c2ffe933c412a8409dee12d75c34b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861c2ffe933c412a8409dee12d75c34b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42317 } } }
I20260806 22:40:14.413491 14678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:14.413770 14677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "861c2ffe933c412a8409dee12d75c34b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861c2ffe933c412a8409dee12d75c34b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42317 } } }
I20260806 22:40:14.414177 14677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:14.416998 14686 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:14.423429 14686 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:14.435428 14686 catalog_manager.cc:1347] Generated new cluster ID: 8f3e5b6c075441298cc58cd3e123438a
I20260806 22:40:14.435619 14686 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:14.461184 14686 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:14.461817 14686 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:14.470453 14686 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 861c2ffe933c412a8409dee12d75c34b: Generated new TSK 0
I20260806 22:40:14.472038 14686 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:14.483234 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 14562
W20260806 22:40:14.484714 14650 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.166:37759
W20260806 22:40:14.485692 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 14562: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:40:14.485975 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 14562
2026-08-06T22:40:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2560 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:14 dist-test-slave-h3ft krb5kdc[14702](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:14 dist-test-slave-h3ft krb5kdc[14702](info): set up 2 sockets
Aug 06 22:40:14 dist-test-slave-h3ft krb5kdc[14702](info): commencing operation
krb5kdc: starting...
W20260806 22:40:15.945071 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.405s user 0.002s sys 0.066s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:40:16 dist-test-slave-h3ft krb5kdc[14702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:40:16.394873 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:40729
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:43037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:40729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:16.834385 14719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:16.834901 14719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:16.835116 14719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:16.843596 14719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:16.843811 14719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:16.843967 14719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:16.844089 14719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:16.844188 14719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:16.858742 14719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:43037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:40729
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:40729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:16.860812 14719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:16.862619 14719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:16.878847 14725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:16.878847 14728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:16.879341 14724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:16.879465 14719 server_base.cc:1034] running on GCE node
I20260806 22:40:16.881006 14719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:16.883248 14719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:16.884575 14719 hybrid_clock.cc:648] HybridClock initialized: now 1786056016884511 us; error 60 us; skew 500 ppm
I20260806 22:40:16.885154 14719 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:16 dist-test-slave-h3ft krb5kdc[14702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056016, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:16.889319 14719 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:40:16.891475 14719 webserver.cc:460] Webserver started at http://127.15.198.190:39209/ using document root <none> and password file <none>
I20260806 22:40:16.892367 14719 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:16.892511 14719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:16.892864 14719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:16.896745 14719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "3fda0ca64b6a4146bffce9126b61f280"
format_stamp: "Formatted at 2026-08-06 22:40:16 on dist-test-slave-h3ft"
server_key: "b6fb55aaae33ad6d545f5e6f7f50ef09"
server_key_iv: "362c7298dd7e675d56edc028cdbce822"
server_key_version: "encryptionkey@0"
I20260806 22:40:16.897620 14719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "3fda0ca64b6a4146bffce9126b61f280"
format_stamp: "Formatted at 2026-08-06 22:40:16 on dist-test-slave-h3ft"
server_key: "b6fb55aaae33ad6d545f5e6f7f50ef09"
server_key_iv: "362c7298dd7e675d56edc028cdbce822"
server_key_version: "encryptionkey@0"
I20260806 22:40:16.905200 14719 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.003s
I20260806 22:40:16.910003 14739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:16.912099 14719 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:40:16.912302 14719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "3fda0ca64b6a4146bffce9126b61f280"
format_stamp: "Formatted at 2026-08-06 22:40:16 on dist-test-slave-h3ft"
server_key: "b6fb55aaae33ad6d545f5e6f7f50ef09"
server_key_iv: "362c7298dd7e675d56edc028cdbce822"
server_key_version: "encryptionkey@0"
I20260806 22:40:16.912534 14719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:16.928258 14719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:16.933960 14719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:16.934278 14719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:16.958153 14719 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:40729
I20260806 22:40:16.958192 14803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:40729 every 8 connection(s)
I20260806 22:40:16.960071 14719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:16.962621 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 14719
I20260806 22:40:16.962863 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:40:16.963460 16154 external_mini_cluster.cc:1428] Setting key 9cd17f80841987477e757445557ac523
I20260806 22:40:16.966362 14807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 22:40:16 dist-test-slave-h3ft krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:40:16.982156 14807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280: Bootstrap starting.
I20260806 22:40:16.988291 14807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:16.990211 14807 log.cc:824] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:16.993785 14813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:16.965966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42483 (local address 127.15.198.190:40729)
0806 22:40:16.967092 (+ 1126us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:16.967109 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:16.967163 (+ 54us) server_negotiation.cc:407] Connection header received
0806 22:40:16.969158 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:16.969189 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:16.969690 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:16.970244 (+ 554us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:16.972116 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:16.973702 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:16.975298 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:16.976155 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:16.976368 (+ 213us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:16.981498 (+ 5130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:16.981518 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:16.981548 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:16.981622 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:16.986883 (+ 5261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:16.988041 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:16.988055 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:16.988070 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:16.988386 (+ 316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:16.989024 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:16.989033 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:16.989037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:16.990012 (+ 975us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:16.990468 (+ 456us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:16.991411 (+ 943us) server_negotiation.cc:299] Negotiation successful
0806 22:40:16.991884 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":793,"thread_start_us":516,"threads_started":1}
I20260806 22:40:16.995183 14807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280: No bootstrap required, opened a new log
I20260806 22:40:17.000072 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:17.006656 14807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fda0ca64b6a4146bffce9126b61f280" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40729 } }
I20260806 22:40:17.007316 14807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:17.007474 14807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fda0ca64b6a4146bffce9126b61f280, State: Initialized, Role: FOLLOWER
I20260806 22:40:17.008308 14807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fda0ca64b6a4146bffce9126b61f280" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40729 } }
I20260806 22:40:17.008603 14807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:17.008772 14807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:17.008973 14807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:17.011879 14807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fda0ca64b6a4146bffce9126b61f280" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40729 } }
I20260806 22:40:17.012603 14807 leader_election.cc:302] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fda0ca64b6a4146bffce9126b61f280; no voters:
I20260806 22:40:17.013442 14807 leader_election.cc:288] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:17.013599 14825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:17.015319 14825 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [term 1 LEADER]: Becoming Leader. State: Replica: 3fda0ca64b6a4146bffce9126b61f280, State: Running, Role: LEADER
I20260806 22:40:17.016119 14825 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fda0ca64b6a4146bffce9126b61f280" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40729 } }
I20260806 22:40:17.016657 14807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:17.020968 14826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fda0ca64b6a4146bffce9126b61f280" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fda0ca64b6a4146bffce9126b61f280" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40729 } } }
I20260806 22:40:17.021477 14826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:17.024608 14813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:17.006220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37479 (local address 127.15.198.190:40729)
0806 22:40:17.006514 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:17.006523 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:17.006776 (+ 253us) server_negotiation.cc:407] Connection header received
0806 22:40:17.007066 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:17.007073 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:17.007163 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:17.007289 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:17.009341 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.010270 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.011800 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:17.012386 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:17.012592 (+ 206us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:17.016155 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:17.016160 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:17.016164 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:17.016204 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:17.020556 (+ 4352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:17.021865 (+ 1309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:17.021872 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:17.021876 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:17.022149 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:17.022902 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:17.022906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:17.022908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:17.023305 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:17.023672 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:17.024061 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 22:40:17.024260 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260806 22:40:17.026127 14828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fda0ca64b6a4146bffce9126b61f280. Latest consensus state: current_term: 1 leader_uuid: "3fda0ca64b6a4146bffce9126b61f280" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fda0ca64b6a4146bffce9126b61f280" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40729 } } }
I20260806 22:40:17.026546 14828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:17.030696 14834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:17.037096 14834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:17.047912 14834 catalog_manager.cc:1347] Generated new cluster ID: 7cc04d7008d44dc8b6123a2787c95fa2
I20260806 22:40:17.048089 14834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:17.071983 14834 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:17.074340 14834 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:17.086951 14834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fda0ca64b6a4146bffce9126b61f280: Generated new TSK 0
I20260806 22:40:17.088460 14834 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:17.106614 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 14719
I20260806 22:40:17.108645 14797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:40:17.214205 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 14719
W20260806 22:40:17.242676 14699 connection.cc:535] client connection to 127.15.198.190:40729 recv error: Network error: failed to read from TLS socket (remote: 127.15.198.190:40729): Connection reset by peer (error 104)
2026-08-06T22:40:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2723 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:17 dist-test-slave-h3ft krb5kdc[14856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:17 dist-test-slave-h3ft krb5kdc[14856](info): set up 2 sockets
Aug 06 22:40:17 dist-test-slave-h3ft krb5kdc[14856](info): commencing operation
krb5kdc: starting...
W20260806 22:40:18.694881 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.431s user 0.000s sys 0.072s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:40:18 dist-test-slave-h3ft krb5kdc[14856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:40:19.143469 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:46089
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:46089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:19.565981 14873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:19.566390 14873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:19.566497 14873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:19.574635 14873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:19.574802 14873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:19.574930 14873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:19.575116 14873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:19.575217 14873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:19.586514 14873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:46089
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:46089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:19.588333 14873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:19.590087 14873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:19.604094 14879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:19.604310 14883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:19.604259 14880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:19.605731 14873 server_base.cc:1034] running on GCE node
I20260806 22:40:19.606873 14873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:19.608742 14873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:19.610009 14873 hybrid_clock.cc:648] HybridClock initialized: now 1786056019609952 us; error 48 us; skew 500 ppm
I20260806 22:40:19.610399 14873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:19 dist-test-slave-h3ft krb5kdc[14856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056019, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:19.617636 14873 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:40:19.619656 14873 webserver.cc:460] Webserver started at http://127.15.198.190:43779/ using document root <none> and password file <none>
I20260806 22:40:19.620548 14873 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:19.620666 14873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:19.621074 14873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:19.624819 14873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "c5bcc32eeea440648e358762f57f5442"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-h3ft"
server_key: "9f1edecacde9970af5787e37a2a2eba4"
server_key_iv: "a6aac84915aeddab1e20c808fb68ccdc"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.625742 14873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "c5bcc32eeea440648e358762f57f5442"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-h3ft"
server_key: "9f1edecacde9970af5787e37a2a2eba4"
server_key_iv: "a6aac84915aeddab1e20c808fb68ccdc"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.632195 14873 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260806 22:40:19.636891 14891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:19.638576 14873 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 22:40:19.638914 14873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "c5bcc32eeea440648e358762f57f5442"
format_stamp: "Formatted at 2026-08-06 22:40:19 on dist-test-slave-h3ft"
server_key: "9f1edecacde9970af5787e37a2a2eba4"
server_key_iv: "a6aac84915aeddab1e20c808fb68ccdc"
server_key_version: "encryptionkey@0"
I20260806 22:40:19.639245 14873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:19.677824 14873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:19.683894 14873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:19.684289 14873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:19.708658 14873 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:46089
I20260806 22:40:19.708663 14959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:46089 every 8 connection(s)
I20260806 22:40:19.710574 14873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:19.715515 14960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:19.720605 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 14873
I20260806 22:40:19.720852 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:40:19.721506 16154 external_mini_cluster.cc:1428] Setting key b534f4e0e7c3bd20df52541d8888c18e
I20260806 22:40:19.729488 14960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442: Bootstrap starting.
I20260806 22:40:19.734033 14960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:19.735641 14960 log.cc:824] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:19.738546 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.723999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46075 (local address 127.15.198.190:46089)
0806 22:40:19.724954 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:19.724969 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:19.725020 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:40:19.727668 (+ 2648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:19.727699 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:19.728193 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:19.728574 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:19.729331 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.730872 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.732985 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.733709 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.733844 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.735568 (+ 1724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:19.735576 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:19.735592 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:19.736478 (+ 886us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:19.736599 (+ 121us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:19.736634 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:19.736824 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:19.737157 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 22:40:19.737391 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":705,"thread_start_us":455,"threads_started":1}
I20260806 22:40:19.739506 14960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442: No bootstrap required, opened a new log
I20260806 22:40:19.743538 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:19.748847 14960 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bcc32eeea440648e358762f57f5442" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46089 } }
I20260806 22:40:19.749668 14960 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:19.749804 14960 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bcc32eeea440648e358762f57f5442, State: Initialized, Role: FOLLOWER
I20260806 22:40:19.750681 14960 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bcc32eeea440648e358762f57f5442" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46089 } }
I20260806 22:40:19.751138 14960 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:19.751410 14960 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:19.751657 14960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:19.755270 14960 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bcc32eeea440648e358762f57f5442" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46089 } }
I20260806 22:40:19.756148 14960 leader_election.cc:302] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5bcc32eeea440648e358762f57f5442; no voters:
I20260806 22:40:19.757062 14960 leader_election.cc:288] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:19.757200 14975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:19.757918 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.749192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44483 (local address 127.15.198.190:46089)
0806 22:40:19.749501 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:19.749510 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:19.749966 (+ 456us) server_negotiation.cc:407] Connection header received
0806 22:40:19.750237 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:19.750244 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:19.750347 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:19.750457 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:19.751263 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.752342 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.754406 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:19.754907 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:19.755099 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:19.756545 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:19.756551 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:19.756554 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:19.756577 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:19.756639 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:19.756662 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:19.756810 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:19.757066 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 22:40:19.757419 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 22:40:19.760465 14960 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:19.762951 14975 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [term 1 LEADER]: Becoming Leader. State: Replica: c5bcc32eeea440648e358762f57f5442, State: Running, Role: LEADER
I20260806 22:40:19.763974 14975 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bcc32eeea440648e358762f57f5442" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46089 } }
I20260806 22:40:19.768271 14978 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5bcc32eeea440648e358762f57f5442. Latest consensus state: current_term: 1 leader_uuid: "c5bcc32eeea440648e358762f57f5442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bcc32eeea440648e358762f57f5442" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46089 } } }
I20260806 22:40:19.768724 14978 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:19.769483 14976 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5bcc32eeea440648e358762f57f5442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bcc32eeea440648e358762f57f5442" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 46089 } } }
I20260806 22:40:19.769799 14976 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:19.770103 14990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:19.782536 14990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:19.799907 14990 catalog_manager.cc:1347] Generated new cluster ID: 77ba9c268aaf46be81df2bfb174f6322
I20260806 22:40:19.800086 14990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:19.824561 14990 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:19.826368 14990 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:19.834817 14990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5bcc32eeea440648e358762f57f5442: Generated new TSK 0
I20260806 22:40:19.836270 14990 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:19.854050 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 14873
W20260806 22:40:19.855468 14954 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46075
W20260806 22:40:19.856389 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 14873: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:40:19.856566 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 14873
W20260806 22:40:19.886309 14852 connection.cc:535] client connection to 127.15.198.190:46089 recv error: Network error: failed to read from TLS socket (remote: 127.15.198.190:46089): Connection reset by peer (error 104)
2026-08-06T22:40:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2643 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:20 dist-test-slave-h3ft krb5kdc[15005](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:20 dist-test-slave-h3ft krb5kdc[15005](info): set up 2 sockets
Aug 06 22:40:20 dist-test-slave-h3ft krb5kdc[15005](info): commencing operation
krb5kdc: starting...
W20260806 22:40:21.256608 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.349s user 0.003s sys 0.068s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:40:21 dist-test-slave-h3ft krb5kdc[15005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:40:21.704684 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:42411
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:35931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:42411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:22.161058 15024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:22.161650 15024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:22.161813 15024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:22.170069 15024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:22.170262 15024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:22.170317 15024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:22.170388 15024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:22.170440 15024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:22.182382 15024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:35931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:42411
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:42411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:22.184269 15024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:22.185997 15024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:22.201102 15032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:22.201043 15036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:22.201305 15033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:22.202699 15024 server_base.cc:1034] running on GCE node
I20260806 22:40:22.204113 15024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:22.206156 15024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:22.207455 15024 hybrid_clock.cc:648] HybridClock initialized: now 1786056022207403 us; error 58 us; skew 500 ppm
I20260806 22:40:22.207995 15024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:22 dist-test-slave-h3ft krb5kdc[15005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056022, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:22.212513 15024 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:40:22.214781 15024 webserver.cc:460] Webserver started at http://127.15.198.190:35269/ using document root <none> and password file <none>
I20260806 22:40:22.215858 15024 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:22.216049 15024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:22.216485 15024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:22.220749 15024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "b54304f3e69f4e01925ffbfaf5dae32e"
format_stamp: "Formatted at 2026-08-06 22:40:22 on dist-test-slave-h3ft"
server_key: "f79729ca541b2686cbef317a7fb5d0f7"
server_key_iv: "0d6423e53963bbe400d14227163c19f6"
server_key_version: "encryptionkey@0"
I20260806 22:40:22.221807 15024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "b54304f3e69f4e01925ffbfaf5dae32e"
format_stamp: "Formatted at 2026-08-06 22:40:22 on dist-test-slave-h3ft"
server_key: "f79729ca541b2686cbef317a7fb5d0f7"
server_key_iv: "0d6423e53963bbe400d14227163c19f6"
server_key_version: "encryptionkey@0"
I20260806 22:40:22.228904 15024 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.004s
I20260806 22:40:22.233552 15046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:22.235450 15024 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:40:22.235690 15024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "b54304f3e69f4e01925ffbfaf5dae32e"
format_stamp: "Formatted at 2026-08-06 22:40:22 on dist-test-slave-h3ft"
server_key: "f79729ca541b2686cbef317a7fb5d0f7"
server_key_iv: "0d6423e53963bbe400d14227163c19f6"
server_key_version: "encryptionkey@0"
I20260806 22:40:22.235954 15024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:22.243193 15024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:22.248854 15024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:22.249228 15024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:22.273669 15024 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:42411
I20260806 22:40:22.273676 15120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:42411 every 8 connection(s)
I20260806 22:40:22.275679 15024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:22.280911 15121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:22.284667 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 15024
I20260806 22:40:22.284912 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:40:22.285482 16154 external_mini_cluster.cc:1428] Setting key ddbd03e07e310cace1c51b50559ffadd
I20260806 22:40:22.297156 15124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.288143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52745 (local address 127.15.198.190:42411)
0806 22:40:22.289206 (+ 1063us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:22.289227 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:22.289290 (+ 63us) server_negotiation.cc:407] Connection header received
0806 22:40:22.292093 (+ 2803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:22.292141 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:22.292801 (+ 660us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:22.293433 (+ 632us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:22.294735 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:22.294744 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:22.294766 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:22.294795 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:22.294897 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:22.294935 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:22.295215 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:22.295357 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 22:40:22.295599 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":445,"threads_started":1}
I20260806 22:40:22.297748 15121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e: Bootstrap starting.
I20260806 22:40:22.301783 15121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:22.301833 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:22.303328 15121 log.cc:824] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:22.307830 15121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e: No bootstrap required, opened a new log
I20260806 22:40:22.311775 15124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.308355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46555 (local address 127.15.198.190:42411)
0806 22:40:22.308617 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:22.308624 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:22.309161 (+ 537us) server_negotiation.cc:407] Connection header received
0806 22:40:22.309342 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:22.309348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:22.309433 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:22.309558 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:22.310744 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:22.310754 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:22.310764 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:22.310786 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:22.310843 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:22.310868 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:22.311053 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:22.311272 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 22:40:22.311356 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 22:40:22.317807 15121 raft_consensus.cc:348] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54304f3e69f4e01925ffbfaf5dae32e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42411 } }
I20260806 22:40:22.318430 15121 raft_consensus.cc:374] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:22.318609 15121 raft_consensus.cc:729] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54304f3e69f4e01925ffbfaf5dae32e, State: Initialized, Role: FOLLOWER
I20260806 22:40:22.319576 15121 consensus_queue.cc:260] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54304f3e69f4e01925ffbfaf5dae32e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42411 } }
I20260806 22:40:22.319958 15121 raft_consensus.cc:388] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:22.320147 15121 raft_consensus.cc:482] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:22.320369 15121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:22.323101 15121 raft_consensus.cc:504] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54304f3e69f4e01925ffbfaf5dae32e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42411 } }
I20260806 22:40:22.323788 15121 leader_election.cc:302] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b54304f3e69f4e01925ffbfaf5dae32e; no voters:
I20260806 22:40:22.324714 15121 leader_election.cc:288] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:22.324885 15137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:22.325819 15137 raft_consensus.cc:686] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [term 1 LEADER]: Becoming Leader. State: Replica: b54304f3e69f4e01925ffbfaf5dae32e, State: Running, Role: LEADER
I20260806 22:40:22.326494 15137 consensus_queue.cc:237] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54304f3e69f4e01925ffbfaf5dae32e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42411 } }
I20260806 22:40:22.327455 15121 sys_catalog.cc:564] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:22.330497 15141 sys_catalog.cc:455] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b54304f3e69f4e01925ffbfaf5dae32e. Latest consensus state: current_term: 1 leader_uuid: "b54304f3e69f4e01925ffbfaf5dae32e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54304f3e69f4e01925ffbfaf5dae32e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42411 } } }
I20260806 22:40:22.330463 15140 sys_catalog.cc:455] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b54304f3e69f4e01925ffbfaf5dae32e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54304f3e69f4e01925ffbfaf5dae32e" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42411 } } }
I20260806 22:40:22.331073 15140 sys_catalog.cc:458] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:22.332937 15141 sys_catalog.cc:458] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:22.334671 15149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:22.343418 15149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:22.358748 15149 catalog_manager.cc:1347] Generated new cluster ID: 43fbea3447d247a48c8b73ad84709b0b
I20260806 22:40:22.358932 15149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:22.388756 15149 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:22.389484 15149 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:22.407388 15149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b54304f3e69f4e01925ffbfaf5dae32e: Generated new TSK 0
I20260806 22:40:22.408710 15149 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:22.426524 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 15024
W20260806 22:40:22.428411 15115 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52745
W20260806 22:40:22.429451 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 15024: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:40:22.429657 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 15024
2026-08-06T22:40:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2573 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:22 dist-test-slave-h3ft krb5kdc[15164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:22 dist-test-slave-h3ft krb5kdc[15164](info): set up 2 sockets
Aug 06 22:40:22 dist-test-slave-h3ft krb5kdc[15164](info): commencing operation
krb5kdc: starting...
W20260806 22:40:23.841118 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.360s user 0.003s sys 0.069s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:40:24 dist-test-slave-h3ft krb5kdc[15164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:40:24.296017 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:45651
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:34471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:45651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:24.732939 15183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:24.733376 15183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:24.733485 15183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:24.741978 15183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:24.742172 15183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:24.742228 15183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:24.742305 15183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:24.742354 15183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:24.754285 15183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:34471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:45651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:45651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:24.756282 15183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:24.757985 15183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:24.773298 15191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:24.773407 15194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:24.773379 15192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:24.775048 15183 server_base.cc:1034] running on GCE node
I20260806 22:40:24.776091 15183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:24.778041 15183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:24.779318 15183 hybrid_clock.cc:648] HybridClock initialized: now 1786056024779252 us; error 50 us; skew 500 ppm
I20260806 22:40:24.779729 15183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:24 dist-test-slave-h3ft krb5kdc[15164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:24.788864 15183 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:40:24.791181 15183 webserver.cc:460] Webserver started at http://127.15.198.190:42905/ using document root <none> and password file <none>
I20260806 22:40:24.792168 15183 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:24.792331 15183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:24.792760 15183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:24.796970 15183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "52ff379fa1274885b91781ca655ff6b4"
format_stamp: "Formatted at 2026-08-06 22:40:24 on dist-test-slave-h3ft"
server_key: "114dc337f06521434d3ecbc81bb2ff11"
server_key_iv: "2260e3e5c63150a3aa074e2c8fa74213"
server_key_version: "encryptionkey@0"
I20260806 22:40:24.798051 15183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "52ff379fa1274885b91781ca655ff6b4"
format_stamp: "Formatted at 2026-08-06 22:40:24 on dist-test-slave-h3ft"
server_key: "114dc337f06521434d3ecbc81bb2ff11"
server_key_iv: "2260e3e5c63150a3aa074e2c8fa74213"
server_key_version: "encryptionkey@0"
I20260806 22:40:24.805540 15183 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 22:40:24.810521 15203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:24.812661 15183 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:40:24.812953 15183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "52ff379fa1274885b91781ca655ff6b4"
format_stamp: "Formatted at 2026-08-06 22:40:24 on dist-test-slave-h3ft"
server_key: "114dc337f06521434d3ecbc81bb2ff11"
server_key_iv: "2260e3e5c63150a3aa074e2c8fa74213"
server_key_version: "encryptionkey@0"
I20260806 22:40:24.813248 15183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:24.827731 15183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:24.833613 15183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:24.834034 15183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:24.867162 15183 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:45651
I20260806 22:40:24.867201 15271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:45651 every 8 connection(s)
I20260806 22:40:24.869140 15183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:24.873752 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 15183
I20260806 22:40:24.874004 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:40:24.874715 16154 external_mini_cluster.cc:1428] Setting key 3b67e91dda4f0b696714e1e23198d53b
I20260806 22:40:24.874917 15276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:24.891065 15276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4: Bootstrap starting.
Aug 06 22:40:24 dist-test-slave-h3ft krb5kdc[15164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:40:24.895784 15276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:24.897277 15276 log.cc:824] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:24.902987 15276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4: No bootstrap required, opened a new log
I20260806 22:40:24.905104 15280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:24.877187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58353 (local address 127.15.198.190:45651)
0806 22:40:24.878195 (+ 1008us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:24.878211 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:24.878264 (+ 53us) server_negotiation.cc:407] Connection header received
0806 22:40:24.880926 (+ 2662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:24.880997 (+ 71us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:24.881576 (+ 579us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:24.882130 (+ 554us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:24.884102 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.885718 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.887229 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.888097 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.888352 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:24.893592 (+ 5240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:24.893603 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:24.893623 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:24.893686 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:24.897501 (+ 3815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.899391 (+ 1890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.899402 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.899413 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.899851 (+ 438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.900498 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.900503 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.900506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.901437 (+ 931us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:24.901827 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:24.902409 (+ 582us) server_negotiation.cc:299] Negotiation successful
0806 22:40:24.903248 (+ 839us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":756,"thread_start_us":446,"threads_started":1}
I20260806 22:40:24.911211 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:24.914213 15276 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ff379fa1274885b91781ca655ff6b4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 45651 } }
I20260806 22:40:24.914958 15276 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:24.915179 15276 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ff379fa1274885b91781ca655ff6b4, State: Initialized, Role: FOLLOWER
I20260806 22:40:24.916095 15276 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ff379fa1274885b91781ca655ff6b4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 45651 } }
I20260806 22:40:24.916498 15276 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:24.916718 15276 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:24.916978 15276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:24.920212 15276 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ff379fa1274885b91781ca655ff6b4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 45651 } }
I20260806 22:40:24.921049 15276 leader_election.cc:302] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52ff379fa1274885b91781ca655ff6b4; no voters:
I20260806 22:40:24.921916 15276 leader_election.cc:288] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:24.922073 15289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:24.923666 15289 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [term 1 LEADER]: Becoming Leader. State: Replica: 52ff379fa1274885b91781ca655ff6b4, State: Running, Role: LEADER
I20260806 22:40:24.924500 15289 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ff379fa1274885b91781ca655ff6b4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 45651 } }
I20260806 22:40:24.925252 15276 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:24.928463 15291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52ff379fa1274885b91781ca655ff6b4. Latest consensus state: current_term: 1 leader_uuid: "52ff379fa1274885b91781ca655ff6b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ff379fa1274885b91781ca655ff6b4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 45651 } } }
I20260806 22:40:24.929316 15291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:24.930519 15290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52ff379fa1274885b91781ca655ff6b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ff379fa1274885b91781ca655ff6b4" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 45651 } } }
I20260806 22:40:24.932910 15290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:24.935719 15296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:24.936090 15280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:24.917291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59871 (local address 127.15.198.190:45651)
0806 22:40:24.917545 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:24.917555 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:24.918212 (+ 657us) server_negotiation.cc:407] Connection header received
0806 22:40:24.918501 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:24.918507 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:24.918617 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:24.918764 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:24.920597 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.921492 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.923418 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:24.924205 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:24.924364 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:24.927829 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:24.927837 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:24.927842 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:24.927890 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:24.932384 (+ 4494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.933512 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.933519 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.933522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.933711 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:24.934169 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:24.934174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:24.934177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:24.934549 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:24.934877 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:24.935362 (+ 485us) server_negotiation.cc:299] Negotiation successful
0806 22:40:24.935645 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 22:40:24.945088 15296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:24.958532 15296 catalog_manager.cc:1347] Generated new cluster ID: bc9bbf94ab2741b68a0a99e8471e80bd
I20260806 22:40:24.958730 15296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:24.978710 15296 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:24.980522 15296 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:24.988452 15296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52ff379fa1274885b91781ca655ff6b4: Generated new TSK 0
I20260806 22:40:24.989635 15296 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:25.001140 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 15183
I20260806 22:40:25.002768 15266 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:40:25.109644 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 15183
2026-08-06T22:40:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2680 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:25 dist-test-slave-h3ft krb5kdc[15321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:25 dist-test-slave-h3ft krb5kdc[15321](info): set up 2 sockets
Aug 06 22:40:25 dist-test-slave-h3ft krb5kdc[15321](info): commencing operation
krb5kdc: starting...
W20260806 22:40:26.517874 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.356s user 0.006s sys 0.068s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:40:26 dist-test-slave-h3ft krb5kdc[15321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:40:26.963917 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:42633
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:42473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:42633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:27.411125 15336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:27.411525 15336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:27.411715 15336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:27.419883 15336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:27.420050 15336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:27.420147 15336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:27.420302 15336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:27.420405 15336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:27.432019 15336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:42473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:42633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:42633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:27.433934 15336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:27.435788 15336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:27.450708 15342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:27.450690 15347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:27.450958 15343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:27.451828 15336 server_base.cc:1034] running on GCE node
I20260806 22:40:27.452868 15336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:27.454695 15336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:27.455920 15336 hybrid_clock.cc:648] HybridClock initialized: now 1786056027455868 us; error 43 us; skew 500 ppm
I20260806 22:40:27.456341 15336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:27 dist-test-slave-h3ft krb5kdc[15321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056027, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:27.460306 15336 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:40:27.462482 15336 webserver.cc:460] Webserver started at http://127.15.198.190:34575/ using document root <none> and password file <none>
I20260806 22:40:27.463495 15336 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:27.463624 15336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:27.463958 15336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:27.467823 15336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "9cfeafd6621941af9d13abb5c1fc2528"
format_stamp: "Formatted at 2026-08-06 22:40:27 on dist-test-slave-h3ft"
server_key: "c3e4ed69ac49505e17cd9a521960fd38"
server_key_iv: "4f372be50d2163312de6c9f919b54306"
server_key_version: "encryptionkey@0"
I20260806 22:40:27.468722 15336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "9cfeafd6621941af9d13abb5c1fc2528"
format_stamp: "Formatted at 2026-08-06 22:40:27 on dist-test-slave-h3ft"
server_key: "c3e4ed69ac49505e17cd9a521960fd38"
server_key_iv: "4f372be50d2163312de6c9f919b54306"
server_key_version: "encryptionkey@0"
I20260806 22:40:27.475766 15336 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260806 22:40:27.480607 15355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:27.482609 15336 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260806 22:40:27.482839 15336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "9cfeafd6621941af9d13abb5c1fc2528"
format_stamp: "Formatted at 2026-08-06 22:40:27 on dist-test-slave-h3ft"
server_key: "c3e4ed69ac49505e17cd9a521960fd38"
server_key_iv: "4f372be50d2163312de6c9f919b54306"
server_key_version: "encryptionkey@0"
I20260806 22:40:27.483144 15336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:27.508667 15336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:27.514856 15336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:27.515394 15336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:27.539527 15336 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:42633
I20260806 22:40:27.539533 15419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:42633 every 8 connection(s)
I20260806 22:40:27.541395 15336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:27.543200 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 15336
I20260806 22:40:27.543471 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:40:27.544142 16154 external_mini_cluster.cc:1428] Setting key e9cec74386637a743de7b078334ad712
I20260806 22:40:27.547875 15421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:27.560758 15427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.546592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56253 (local address 127.15.198.190:42633)
0806 22:40:27.547597 (+ 1005us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.547613 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.547665 (+ 52us) server_negotiation.cc:407] Connection header received
0806 22:40:27.549686 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.549716 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.550222 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.550710 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.551557 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.553092 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.554595 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.555854 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.556095 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.557730 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.557738 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.557758 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:27.558473 (+ 715us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.558574 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:27.558604 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.558772 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.559050 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.559432 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":469,"threads_started":1}
I20260806 22:40:27.562984 15421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528: Bootstrap starting.
I20260806 22:40:27.565362 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:27.567360 15421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:27.568812 15421 log.cc:824] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:27.572712 15421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528: No bootstrap required, opened a new log
I20260806 22:40:27.578336 15427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:27.570722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52415 (local address 127.15.198.190:42633)
0806 22:40:27.570976 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:27.570985 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:27.571482 (+ 497us) server_negotiation.cc:407] Connection header received
0806 22:40:27.571757 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:27.571762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:27.571900 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:27.572033 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:27.572735 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.573688 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.575294 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:27.575867 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:27.576041 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:27.577393 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:27.577399 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:27.577403 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:27.577425 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:27.577478 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:27.577500 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:27.577649 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:27.577841 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 22:40:27.577961 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:40:27.582096 15421 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfeafd6621941af9d13abb5c1fc2528" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42633 } }
I20260806 22:40:27.582595 15421 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:27.582739 15421 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cfeafd6621941af9d13abb5c1fc2528, State: Initialized, Role: FOLLOWER
I20260806 22:40:27.583598 15421 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfeafd6621941af9d13abb5c1fc2528" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42633 } }
I20260806 22:40:27.583874 15421 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:27.584049 15421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:27.584244 15421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:27.587478 15421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfeafd6621941af9d13abb5c1fc2528" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42633 } }
I20260806 22:40:27.588133 15421 leader_election.cc:302] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cfeafd6621941af9d13abb5c1fc2528; no voters:
I20260806 22:40:27.588923 15421 leader_election.cc:288] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:27.589066 15436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:27.589986 15436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [term 1 LEADER]: Becoming Leader. State: Replica: 9cfeafd6621941af9d13abb5c1fc2528, State: Running, Role: LEADER
I20260806 22:40:27.590783 15436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfeafd6621941af9d13abb5c1fc2528" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42633 } }
I20260806 22:40:27.591854 15421 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:27.594429 15437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cfeafd6621941af9d13abb5c1fc2528" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfeafd6621941af9d13abb5c1fc2528" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42633 } } }
I20260806 22:40:27.594461 15438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cfeafd6621941af9d13abb5c1fc2528. Latest consensus state: current_term: 1 leader_uuid: "9cfeafd6621941af9d13abb5c1fc2528" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfeafd6621941af9d13abb5c1fc2528" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42633 } } }
I20260806 22:40:27.595072 15438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:27.595072 15437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:27.597913 15442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:27.604667 15442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:27.616189 15442 catalog_manager.cc:1347] Generated new cluster ID: 45625ceb5bb44b6e9dd8729d2deb3b1e
I20260806 22:40:27.616362 15442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:27.642273 15442 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:27.644830 15442 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:27.660188 15442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cfeafd6621941af9d13abb5c1fc2528: Generated new TSK 0
I20260806 22:40:27.661818 15442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:27.679509 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 15336
W20260806 22:40:27.681469 15415 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56253
W20260806 22:40:27.682471 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 15336: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:40:27.682680 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 15336
2026-08-06T22:40:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2573 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:27 dist-test-slave-h3ft krb5kdc[15463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:27 dist-test-slave-h3ft krb5kdc[15463](info): set up 2 sockets
Aug 06 22:40:27 dist-test-slave-h3ft krb5kdc[15463](info): commencing operation
krb5kdc: starting...
W20260806 22:40:29.162864 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.429s user 0.000s sys 0.071s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:40:29 dist-test-slave-h3ft krb5kdc[15463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:40:29.625234 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:42769
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:34349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:42769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:30.088049 15482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:30.088474 15482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:30.088663 15482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:30.097841 15482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:30.098068 15482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:30.098229 15482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:30.098402 15482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:30.098484 15482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:30.111382 15482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:34349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:42769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:42769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:30.113386 15482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:30.115345 15482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:30.130515 15488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:30.130635 15490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:30.130949 15493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:30.131520 15482 server_base.cc:1034] running on GCE node
I20260806 22:40:30.132596 15482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:30.134529 15482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:30.135808 15482 hybrid_clock.cc:648] HybridClock initialized: now 1786056030135746 us; error 52 us; skew 500 ppm
I20260806 22:40:30.136268 15482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:30 dist-test-slave-h3ft krb5kdc[15463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056030, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:30.145619 15482 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:40:30.147892 15482 webserver.cc:460] Webserver started at http://127.15.198.190:37405/ using document root <none> and password file <none>
I20260806 22:40:30.148875 15482 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:30.149051 15482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:30.149456 15482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:30.153548 15482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "3e831e54a706462f8cb40634426bd928"
format_stamp: "Formatted at 2026-08-06 22:40:30 on dist-test-slave-h3ft"
server_key: "695ea66c0dc3f6e706363a67c1beb9e2"
server_key_iv: "8e628abe5fe2227150eff68c53633f38"
server_key_version: "encryptionkey@0"
I20260806 22:40:30.154533 15482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "3e831e54a706462f8cb40634426bd928"
format_stamp: "Formatted at 2026-08-06 22:40:30 on dist-test-slave-h3ft"
server_key: "695ea66c0dc3f6e706363a67c1beb9e2"
server_key_iv: "8e628abe5fe2227150eff68c53633f38"
server_key_version: "encryptionkey@0"
I20260806 22:40:30.161479 15482 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260806 22:40:30.166481 15501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:30.168638 15482 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 22:40:30.168931 15482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "3e831e54a706462f8cb40634426bd928"
format_stamp: "Formatted at 2026-08-06 22:40:30 on dist-test-slave-h3ft"
server_key: "695ea66c0dc3f6e706363a67c1beb9e2"
server_key_iv: "8e628abe5fe2227150eff68c53633f38"
server_key_version: "encryptionkey@0"
I20260806 22:40:30.169442 15482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:30.177161 15482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:30.182696 15482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:30.183149 15482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:30.208182 15482 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:42769
I20260806 22:40:30.208204 15567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:42769 every 8 connection(s)
I20260806 22:40:30.210476 15482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:30.211043 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 15482
I20260806 22:40:30.211282 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:40:30.212055 16154 external_mini_cluster.cc:1428] Setting key 43748c4627e9dccd2c1c104deb9493c8
I20260806 22:40:30.216548 15569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:30.222560 15572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:30.215184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35491 (local address 127.15.198.190:42769)
0806 22:40:30.216160 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:30.216175 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:30.216230 (+ 55us) server_negotiation.cc:407] Connection header received
0806 22:40:30.218222 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:30.218251 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:30.218660 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:30.219171 (+ 511us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:30.220528 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:30.220536 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:30.220551 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:30.220576 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:30.220662 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:30.220690 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:30.220993 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:30.221159 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 22:40:30.221383 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":409,"threads_started":1}
I20260806 22:40:30.227303 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:30.230845 15569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928: Bootstrap starting.
I20260806 22:40:30.235539 15569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:30.235870 15572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:30.232592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55697 (local address 127.15.198.190:42769)
0806 22:40:30.232835 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:30.232849 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:30.233439 (+ 590us) server_negotiation.cc:407] Connection header received
0806 22:40:30.233703 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:30.233707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:30.233771 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:30.233891 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:30.235105 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:30.235111 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:30.235116 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:40:30.235142 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:30.235193 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:40:30.235211 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:30.235348 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:30.235483 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 22:40:30.235563 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:40:30.237149 15569 log.cc:824] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:30.240617 15569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928: No bootstrap required, opened a new log
I20260806 22:40:30.249061 15569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e831e54a706462f8cb40634426bd928" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42769 } }
I20260806 22:40:30.249611 15569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:30.249740 15569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e831e54a706462f8cb40634426bd928, State: Initialized, Role: FOLLOWER
I20260806 22:40:30.250492 15569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e831e54a706462f8cb40634426bd928" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42769 } }
I20260806 22:40:30.250789 15569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:30.250977 15569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:30.251263 15569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:30.254374 15569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e831e54a706462f8cb40634426bd928" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42769 } }
I20260806 22:40:30.255128 15569 leader_election.cc:302] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e831e54a706462f8cb40634426bd928; no voters:
I20260806 22:40:30.255884 15569 leader_election.cc:288] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:30.256178 15583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:30.257025 15583 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [term 1 LEADER]: Becoming Leader. State: Replica: 3e831e54a706462f8cb40634426bd928, State: Running, Role: LEADER
I20260806 22:40:30.257722 15583 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e831e54a706462f8cb40634426bd928" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42769 } }
I20260806 22:40:30.258723 15569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:30.262584 15584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e831e54a706462f8cb40634426bd928" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e831e54a706462f8cb40634426bd928" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42769 } } }
I20260806 22:40:30.262547 15585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e831e54a706462f8cb40634426bd928. Latest consensus state: current_term: 1 leader_uuid: "3e831e54a706462f8cb40634426bd928" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e831e54a706462f8cb40634426bd928" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 42769 } } }
I20260806 22:40:30.263229 15584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:30.263319 15585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:30.264364 15591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:30.272739 15591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:30.285349 15591 catalog_manager.cc:1347] Generated new cluster ID: 326b26d8132d41e7b615ab6f16e11215
I20260806 22:40:30.285531 15591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:30.301517 15591 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:30.302177 15591 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:30.312484 15591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e831e54a706462f8cb40634426bd928: Generated new TSK 0
I20260806 22:40:30.313870 15591 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:30.323040 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 15482
W20260806 22:40:30.325294 15563 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35491
W20260806 22:40:30.326309 16154 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 15482: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 22:40:30.326521 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 15482
2026-08-06T22:40:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2644 ms)
[----------] 18 tests from AuthTokenIssuingTest (48010 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:30 dist-test-slave-h3ft krb5kdc[15614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:30 dist-test-slave-h3ft krb5kdc[15614](info): set up 2 sockets
Aug 06 22:40:30 dist-test-slave-h3ft krb5kdc[15614](info): commencing operation
krb5kdc: starting...
W20260806 22:40:31.746424 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.366s user 0.005s sys 0.067s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:40:32 dist-test-slave-h3ft krb5kdc[15614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:40:32.203462 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:40509
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:37811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:40509,127.15.198.189:42677,127.15.198.188:34055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:32.652693 15629 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:40:32.653190 15629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:32.653303 15629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:32.653474 15629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:32.662012 15629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:32.662154 15629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:32.662266 15629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:32.662372 15629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:32.662498 15629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:32.674494 15629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:37811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:40509,127.15.198.189:42677,127.15.198.188:34055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:40509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:32.676344 15629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:32.678048 15629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:32.692696 15634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:32.692740 15639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:32.692696 15637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:32.693900 15629 server_base.cc:1034] running on GCE node
I20260806 22:40:32.695118 15629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:32.697113 15629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:32.698391 15629 hybrid_clock.cc:648] HybridClock initialized: now 1786056032698307 us; error 52 us; skew 500 ppm
I20260806 22:40:32.698836 15629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:32 dist-test-slave-h3ft krb5kdc[15614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:32.703115 15629 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:40:32.705070 15629 webserver.cc:460] Webserver started at http://127.15.198.190:45337/ using document root <none> and password file <none>
I20260806 22:40:32.706051 15629 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:32.706188 15629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:32.706609 15629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:32.710608 15629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "e178c562f72343bc8445cc714b923560"
format_stamp: "Formatted at 2026-08-06 22:40:32 on dist-test-slave-h3ft"
server_key: "fba66aa77c577ec388c0aa966abf1fac"
server_key_iv: "fd66bcd91cc9c573fd59075527d51677"
server_key_version: "encryptionkey@0"
I20260806 22:40:32.711652 15629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "e178c562f72343bc8445cc714b923560"
format_stamp: "Formatted at 2026-08-06 22:40:32 on dist-test-slave-h3ft"
server_key: "fba66aa77c577ec388c0aa966abf1fac"
server_key_iv: "fd66bcd91cc9c573fd59075527d51677"
server_key_version: "encryptionkey@0"
I20260806 22:40:32.719578 15629 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 22:40:32.724658 15651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:32.726665 15629 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260806 22:40:32.726954 15629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "e178c562f72343bc8445cc714b923560"
format_stamp: "Formatted at 2026-08-06 22:40:32 on dist-test-slave-h3ft"
server_key: "fba66aa77c577ec388c0aa966abf1fac"
server_key_iv: "fd66bcd91cc9c573fd59075527d51677"
server_key_version: "encryptionkey@0"
I20260806 22:40:32.727332 15629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:32.743588 15629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:32.749303 15629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:32.749723 15629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:32.772679 15629 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:40509
I20260806 22:40:32.772701 15723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:40509 every 8 connection(s)
I20260806 22:40:32.774492 15629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:32.779399 15724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:32.781852 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 15629
I20260806 22:40:32.782146 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:40:32.782979 16154 external_mini_cluster.cc:1428] Setting key d18c408d567d54e9a2ea80bc40953586
I20260806 22:40:32.787889 15724 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:40:32 dist-test-slave-h3ft krb5kdc[15614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:40:32.819212 15728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.792095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39786 (local address 127.15.198.190:40509)
0806 22:40:32.794432 (+ 2337us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:32.794448 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:32.794523 (+ 75us) server_negotiation.cc:407] Connection header received
0806 22:40:32.796535 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:32.796543 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:32.797005 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:32.797300 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:32.799767 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.801209 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.804243 (+ 3034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.805001 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.805283 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.810207 (+ 4924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:32.810217 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:32.810236 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:32.810275 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:32.813698 (+ 3423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:32.814554 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:32.814563 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:32.814565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:32.814739 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:32.815226 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:32.815235 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:32.815238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:32.815901 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:32.816156 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:32.816782 (+ 626us) server_negotiation.cc:299] Negotiation successful
0806 22:40:32.817234 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":2066,"thread_start_us":1759,"threads_started":1}
I20260806 22:40:32.819232 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.792094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:40509 (local address 127.0.0.1:39786)
0806 22:40:32.793305 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:32.793369 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:32.795999 (+ 2630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:32.797391 (+ 1392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:32.797405 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:32.798645 (+ 1240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:32.799450 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.799495 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.801446 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.801456 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.803990 (+ 2534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:32.804019 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:32.805329 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:32.805338 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:32.805523 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:32.806411 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:32.806441 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:32.809879 (+ 3438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:32.813872 (+ 3993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:32.813888 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:32.813904 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:32.814393 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:32.814861 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:32.814867 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:32.814873 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:32.815112 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:32.816043 (+ 931us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:32.816056 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:32.816342 (+ 286us) client_negotiation.cc:769] Sending connection context
0806 22:40:32.816629 (+ 287us) client_negotiation.cc:240] Negotiation successful
0806 22:40:32.817371 (+ 742us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"mutex_wait_us":171,"thread_start_us":467,"threads_started":1}
I20260806 22:40:32.823518 15724 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:40:32.824970 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.824460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:42677 (local address 127.0.0.1:38108)
0806 22:40:32.824670 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:32.824812 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:42677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 22:40:32.825307 15652 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.198.189:42677: connect: Connection refused (error 111)
I20260806 22:40:32.827234 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.826761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:42677 (local address 127.0.0.1:38120)
0806 22:40:32.826933 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:32.827068 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:42677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 22:40:32.827448 15724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.189:42677: Network error: Client connection negotiation failed: client connection to 127.15.198.189:42677: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.15.198.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.189@KRBTEST.COM" created.
I20260806 22:40:32.876888 15724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } attempt: 1
I20260806 22:40:32.878485 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.877869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:42677 (local address 127.0.0.1:38122)
0806 22:40:32.878126 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:32.878281 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:42677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 22:40:32.879673 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.879246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:42677 (local address 127.0.0.1:38138)
0806 22:40:32.879412 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:32.879507 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:42677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 22:40:32.879891 15724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.189:42677: Network error: Client connection negotiation failed: client connection to 127.15.198.189:42677: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.189.keytab.
Entry for principal kudu/127.15.198.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.189.keytab.
I20260806 22:40:32.883899 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.189:42677
--webserver_interface=127.15.198.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.189.keytab
--principal=kudu/127.15.198.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:37811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:40509,127.15.198.189:42677,127.15.198.188:34055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:40:32.948225 15724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } attempt: 2
I20260806 22:40:32.950141 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.949381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:42677 (local address 127.0.0.1:38144)
0806 22:40:32.949750 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:32.949894 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:42677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260806 22:40:32.951578 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:32.950978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:42677 (local address 127.0.0.1:38146)
0806 22:40:32.951255 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:32.951355 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:42677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
W20260806 22:40:32.951959 15724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.189:42677: Network error: Client connection negotiation failed: client connection to 127.15.198.189:42677: connect: Connection refused (error 111)
I20260806 22:40:33.043277 15724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } attempt: 3
I20260806 22:40:33.044986 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.044257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:42677 (local address 127.0.0.1:38152)
0806 22:40:33.044568 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.044736 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:42677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260806 22:40:33.046327 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.045836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:42677 (local address 127.0.0.1:38154)
0806 22:40:33.046018 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.046125 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:42677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 22:40:33.046556 15724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.189:42677: Network error: Client connection negotiation failed: client connection to 127.15.198.189:42677: connect: Connection refused (error 111)
I20260806 22:40:33.189898 15724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } attempt: 4
I20260806 22:40:33.191931 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.190994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:42677 (local address 127.0.0.1:38158)
0806 22:40:33.191492 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.191660 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:42677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210}
I20260806 22:40:33.193455 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.192953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:42677 (local address 127.0.0.1:38174)
0806 22:40:33.193148 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.193265 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:42677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 22:40:33.193711 15724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.189:42677: Network error: Client connection negotiation failed: client connection to 127.15.198.189:42677: connect: Connection refused (error 111)
W20260806 22:40:33.325114 15735 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:40:33.325543 15735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:33.325623 15735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:33.325718 15735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:33.334189 15735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:33.334307 15735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:33.334372 15735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:33.334429 15735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:33.334507 15735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:33.345965 15735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:37811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:40509,127.15.198.189:42677,127.15.198.188:34055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.189.keytab
--principal=kudu/127.15.198.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.189:42677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.198.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:33.347853 15735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:33.349550 15735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:33.363711 15741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:33.363705 15743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:33.363777 15740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:33.364889 15735 server_base.cc:1034] running on GCE node
I20260806 22:40:33.366005 15735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:33.367992 15735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:33.369302 15735 hybrid_clock.cc:648] HybridClock initialized: now 1786056033369234 us; error 42 us; skew 500 ppm
I20260806 22:40:33.369752 15735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:33 dist-test-slave-h3ft krb5kdc[15614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056033, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:33.373865 15735 init.cc:377] Logged in from keytab as kudu/127.15.198.189@KRBTEST.COM (short username kudu)
I20260806 22:40:33.375833 15735 webserver.cc:460] Webserver started at http://127.15.198.189:40289/ using document root <none> and password file <none>
I20260806 22:40:33.376747 15735 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:33.376897 15735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:33.377300 15735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:33.381218 15735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-1/data/instance:
uuid: "093a918807294901bc6d8931de2d4477"
format_stamp: "Formatted at 2026-08-06 22:40:33 on dist-test-slave-h3ft"
server_key: "ab6aa7d58d1680b695be4d78a34aa08c"
server_key_iv: "d9171ce16372c6b1bfe78120a2c28630"
server_key_version: "encryptionkey@0"
I20260806 22:40:33.382151 15735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-1/wal/instance:
uuid: "093a918807294901bc6d8931de2d4477"
format_stamp: "Formatted at 2026-08-06 22:40:33 on dist-test-slave-h3ft"
server_key: "ab6aa7d58d1680b695be4d78a34aa08c"
server_key_iv: "d9171ce16372c6b1bfe78120a2c28630"
server_key_version: "encryptionkey@0"
I20260806 22:40:33.388513 15735 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 22:40:33.393005 15751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:33.394804 15735 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260806 22:40:33.395114 15735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-1/wal
uuid: "093a918807294901bc6d8931de2d4477"
format_stamp: "Formatted at 2026-08-06 22:40:33 on dist-test-slave-h3ft"
server_key: "ab6aa7d58d1680b695be4d78a34aa08c"
server_key_iv: "d9171ce16372c6b1bfe78120a2c28630"
server_key_version: "encryptionkey@0"
I20260806 22:40:33.395391 15735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:33.417027 15735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:33.423522 15735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:33.423898 15735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:33.446426 15735 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.189:42677
I20260806 22:40:33.446450 15819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.189:42677 every 8 connection(s)
I20260806 22:40:33.448324 15735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-1/data/info.pb
I20260806 22:40:33.451258 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 15735
I20260806 22:40:33.451495 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-1/wal/instance
I20260806 22:40:33.452036 16154 external_mini_cluster.cc:1428] Setting key 81408dffa73caa9cbf94675289608aa6
I20260806 22:40:33.453622 15820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:33.461835 15820 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:40:33 dist-test-slave-h3ft krb5kdc[15614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056033, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.189@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:40:33.489341 15823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.465253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39802 (local address 127.15.198.190:40509)
0806 22:40:33.465872 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:33.465878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:33.467210 (+ 1332us) server_negotiation.cc:407] Connection header received
0806 22:40:33.469470 (+ 2260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:33.469477 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:33.469608 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:33.469756 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:33.471893 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:33.472733 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:33.475581 (+ 2848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:33.476085 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:33.476215 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:33.481708 (+ 5493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:33.481714 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:33.481718 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:33.481784 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:33.485110 (+ 3326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:33.486104 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:33.486109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:33.486111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:33.486261 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:33.486776 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:33.486781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:33.486783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:33.487201 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:33.487392 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:33.488438 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0806 22:40:33.488774 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":341,"threads_started":1}
I20260806 22:40:33.490360 15824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.466022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:40509 (local address 127.0.0.1:39802)
0806 22:40:33.466956 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.467064 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:33.469124 (+ 2060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:33.470081 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:33.470091 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:33.470761 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:33.471584 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:33.471615 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:33.472932 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:33.472941 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:33.475381 (+ 2440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:33.475403 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:33.476416 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:33.476430 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:33.476845 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:33.477784 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:33.477824 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:33.481338 (+ 3514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:33.485329 (+ 3991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:33.485342 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:33.485370 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:33.485941 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:33.486387 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:33.486392 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:33.486396 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:33.486642 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:33.487351 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:33.487361 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:33.487876 (+ 515us) client_negotiation.cc:769] Sending connection context
0806 22:40:33.488255 (+ 379us) client_negotiation.cc:240] Negotiation successful
0806 22:40:33.488832 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":409,"threads_started":1}
I20260806 22:40:33.492857 15820 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:40:33.492992 15724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } attempt: 5
WARNING: no policy specified for kudu/127.15.198.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.188@KRBTEST.COM" created.
Aug 06 22:40:33 dist-test-slave-h3ft krb5kdc[15614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056033, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.189@KRBTEST.COM for kudu/127.15.198.189@KRBTEST.COM
Aug 06 22:40:33 dist-test-slave-h3ft krb5kdc[15614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for kudu/127.15.198.189@KRBTEST.COM
I20260806 22:40:33.514237 15824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.493896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:42677 (local address 127.0.0.1:38192)
0806 22:40:33.494275 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.494293 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:33.494401 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:33.495930 (+ 1529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:33.495934 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:33.496487 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:33.496644 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:33.496660 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:33.499294 (+ 2634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:33.499311 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:33.500846 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:33.500865 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:33.502431 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:33.502437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:33.502701 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:33.503577 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:33.503607 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:33.506289 (+ 2682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:33.510934 (+ 4645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:33.510941 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:33.510945 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:33.511512 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:33.511939 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:33.511943 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:33.511946 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:33.512135 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:33.513042 (+ 907us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:33.513046 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:33.513224 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 22:40:33.513374 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 22:40:33.513723 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196}
I20260806 22:40:33.514288 15829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.494205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38192 (local address 127.15.198.189:42677)
0806 22:40:33.495384 (+ 1179us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:33.495398 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:33.495443 (+ 45us) server_negotiation.cc:407] Connection header received
0806 22:40:33.495512 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:33.495522 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:33.495766 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:33.495949 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:33.496840 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:33.499092 (+ 2252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:33.501013 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:33.502251 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:33.503144 (+ 893us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:33.506590 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:33.506611 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:33.506637 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:33.506690 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:33.510761 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:33.511658 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:33.511673 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:33.511676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:33.511817 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:33.512307 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:33.512311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:33.512314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:33.512898 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:33.513119 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:33.513563 (+ 444us) server_negotiation.cc:299] Negotiation successful
0806 22:40:33.513828 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":312,"server-negotiator.queue_time_us":890,"thread_start_us":625,"threads_started":1}
I20260806 22:40:33.515859 15830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.494812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38176 (local address 127.15.198.189:42677)
0806 22:40:33.495706 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:33.495714 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:33.495740 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:40:33.495791 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:33.495798 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:33.495871 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:33.496840 (+ 969us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:33.497192 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:33.499092 (+ 1900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:33.501025 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:33.502251 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:33.502399 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:33.507283 (+ 4884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:33.507292 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:33.507300 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:33.507347 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:33.511721 (+ 4374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:33.512732 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:33.512740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:33.512746 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:33.512916 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:33.513425 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:33.513432 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:33.513437 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:33.513827 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:33.513985 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:33.515199 (+ 1214us) server_negotiation.cc:299] Negotiation successful
0806 22:40:33.515416 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":752,"thread_start_us":500,"threads_started":1}
I20260806 22:40:33.516343 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.493794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:42677 (local address 127.0.0.1:38176)
0806 22:40:33.494148 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.494175 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:33.494316 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:33.496094 (+ 1778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:33.496102 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:33.496703 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:33.496997 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:33.497011 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:33.499344 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:33.499350 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:33.500854 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:33.500871 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:33.502451 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:33.502459 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:33.502788 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:33.503759 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:33.503793 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:33.506917 (+ 3124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:33.511914 (+ 4997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:33.511922 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:33.511926 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:33.512594 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:33.513079 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:33.513086 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:33.513092 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:33.513278 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:33.514082 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:33.514090 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:33.514301 (+ 211us) client_negotiation.cc:769] Sending connection context
0806 22:40:33.515108 (+ 807us) client_negotiation.cc:240] Negotiation successful
0806 22:40:33.515543 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260806 22:40:33.517707 15820 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:40:33.518195 15724 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:40:33.519239 15824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.518582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46618)
0806 22:40:33.518794 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.518964 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 22:40:33.519351 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.518891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46632)
0806 22:40:33.519087 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.519182 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 22:40:33.519565 15756 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
I20260806 22:40:33.520397 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.519941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46644)
0806 22:40:33.520099 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.520214 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 22:40:33.520642 15724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:34055: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
I20260806 22:40:33.521703 15824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.521352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46650)
0806 22:40:33.521499 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.521590 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 22:40:33.521977 15820 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:34055: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.188.keytab.
Entry for principal kudu/127.15.198.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.188.keytab.
W20260806 22:40:33.555558 16154 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.198.188: real 0.103s user 0.004s sys 0.037s
I20260806 22:40:33.556480 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.188:34055
--webserver_interface=127.15.198.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.188.keytab
--principal=kudu/127.15.198.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:37811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:40509,127.15.198.189:42677,127.15.198.188:34055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:40:33.571396 15820 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } attempt: 1
I20260806 22:40:33.571899 15724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } attempt: 1
I20260806 22:40:33.573170 15824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.572494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46664)
0806 22:40:33.572784 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.572953 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260806 22:40:33.573576 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.572671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46676)
0806 22:40:33.572947 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.573067 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 22:40:33.574429 15824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.573925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46686)
0806 22:40:33.574119 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.574235 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 22:40:33.574666 15820 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:34055: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
I20260806 22:40:33.574990 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.574377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46688)
0806 22:40:33.574728 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.574836 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 22:40:33.575273 15724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:34055: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
I20260806 22:40:33.642984 15820 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } attempt: 2
I20260806 22:40:33.643525 15724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } attempt: 2
I20260806 22:40:33.644814 15824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.644098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46696)
0806 22:40:33.644383 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.644545 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260806 22:40:33.644999 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.644289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46698)
0806 22:40:33.644649 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.644761 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201}
I20260806 22:40:33.646251 15824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.645831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46710)
0806 22:40:33.646049 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.646126 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260806 22:40:33.646265 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.645815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46702)
0806 22:40:33.646010 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.646111 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260806 22:40:33.646525 15820 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:34055: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
W20260806 22:40:33.646670 15724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:34055: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
I20260806 22:40:33.737792 15820 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } attempt: 3
I20260806 22:40:33.739521 15824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.738694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46720)
0806 22:40:33.739072 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.739234 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 22:40:33.741012 15824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.740419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46730)
0806 22:40:33.740712 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.740819 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
W20260806 22:40:33.741330 15820 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:34055: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
I20260806 22:40:33.752967 15724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } attempt: 3
I20260806 22:40:33.754932 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.754081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46738)
0806 22:40:33.754416 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.754600 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260806 22:40:33.756531 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.755981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46740)
0806 22:40:33.756225 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.756352 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260806 22:40:33.756798 15724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:34055: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
I20260806 22:40:33.884652 15820 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } attempt: 4
I20260806 22:40:33.886544 15824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.885774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46746)
0806 22:40:33.886106 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.886257 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260806 22:40:33.887989 15824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.887564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46758)
0806 22:40:33.887732 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.887823 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 22:40:33.888391 15820 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:34055: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
I20260806 22:40:33.934072 15724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } attempt: 4
I20260806 22:40:33.935873 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.935138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46774)
0806 22:40:33.935443 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.935593 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 22:40:33.937282 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:33.936758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46784)
0806 22:40:33.936949 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:33.937064 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 22:40:33.937584 15724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:34055: Network error: Client connection negotiation failed: client connection to 127.15.198.188:34055: connect: Connection refused (error 111)
W20260806 22:40:33.997936 15837 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:40:33.998414 15837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:33.998546 15837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:33.998689 15837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:34.007061 15837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:34.007187 15837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:34.007298 15837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:34.007371 15837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:34.007464 15837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:34.018893 15837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:37811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:40509,127.15.198.189:42677,127.15.198.188:34055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.188.keytab
--principal=kudu/127.15.198.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.188:34055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.198.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:34.020763 15837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:34.022535 15837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:34.037535 15843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:34.037433 15845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:34.038401 15837 server_base.cc:1034] running on GCE node
W20260806 22:40:34.038440 15847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:34.039603 15837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:34.041417 15837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:34.042695 15837 hybrid_clock.cc:648] HybridClock initialized: now 1786056034042619 us; error 46 us; skew 500 ppm
I20260806 22:40:34.043154 15837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:34 dist-test-slave-h3ft krb5kdc[15614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056034, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:34.046962 15837 init.cc:377] Logged in from keytab as kudu/127.15.198.188@KRBTEST.COM (short username kudu)
I20260806 22:40:34.048899 15837 webserver.cc:460] Webserver started at http://127.15.198.188:34871/ using document root <none> and password file <none>
I20260806 22:40:34.049801 15837 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:34.049942 15837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:34.050321 15837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:34.054260 15837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-2/data/instance:
uuid: "935a89a3f5694b9090fc4f66f3142811"
format_stamp: "Formatted at 2026-08-06 22:40:34 on dist-test-slave-h3ft"
server_key: "f4ef95cc7b5731ee279443bab1b2e378"
server_key_iv: "c662198a87ddad35a10feb4057be1c15"
server_key_version: "encryptionkey@0"
I20260806 22:40:34.055301 15837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-2/wal/instance:
uuid: "935a89a3f5694b9090fc4f66f3142811"
format_stamp: "Formatted at 2026-08-06 22:40:34 on dist-test-slave-h3ft"
server_key: "f4ef95cc7b5731ee279443bab1b2e378"
server_key_iv: "c662198a87ddad35a10feb4057be1c15"
server_key_version: "encryptionkey@0"
I20260806 22:40:34.061772 15837 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260806 22:40:34.066031 15856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:34.067921 15837 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 22:40:34.068171 15837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-2/wal
uuid: "935a89a3f5694b9090fc4f66f3142811"
format_stamp: "Formatted at 2026-08-06 22:40:34 on dist-test-slave-h3ft"
server_key: "f4ef95cc7b5731ee279443bab1b2e378"
server_key_iv: "c662198a87ddad35a10feb4057be1c15"
server_key_version: "encryptionkey@0"
I20260806 22:40:34.068466 15837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:34.097016 15837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:34.103399 15837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:34.103783 15837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:34.126847 15837 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.188:34055
I20260806 22:40:34.126865 15929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.188:34055 every 8 connection(s)
I20260806 22:40:34.128670 15837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-2/data/info.pb
I20260806 22:40:34.133787 15931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:34.135118 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 15837
I20260806 22:40:34.135372 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055595907839-16154-0/minicluster-data/master-2/wal/instance
I20260806 22:40:34.136018 16154 external_mini_cluster.cc:1428] Setting key dec5bfe6517d1bc40dbe69909b98c952
I20260806 22:40:34.145506 15931 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:40:34 dist-test-slave-h3ft krb5kdc[15614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.188@KRBTEST.COM
Aug 06 22:40:34 dist-test-slave-h3ft krb5kdc[15614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056034, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.188@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:40:34.164193 15935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:34.138447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46794 (local address 127.15.198.188:34055)
0806 22:40:34.139454 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.139472 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.139531 (+ 59us) server_negotiation.cc:407] Connection header received
0806 22:40:34.141945 (+ 2414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.141974 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.142473 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.142868 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.144977 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.146485 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.148412 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.149699 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.149933 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:34.154900 (+ 4967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:34.154911 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:34.154924 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:34.154971 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:34.158417 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.159311 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.159319 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.159327 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.159537 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.159942 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.159949 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.159954 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.160907 (+ 953us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:34.161328 (+ 421us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:34.161743 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 22:40:34.162277 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":775,"thread_start_us":471,"threads_started":1}
I20260806 22:40:34.167178 15937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:34.148937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:40509 (local address 127.0.0.1:39810)
0806 22:40:34.150873 (+ 1936us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:34.150921 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:34.151164 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:34.151677 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:34.151682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:34.152375 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:34.152662 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:34.152678 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.153999 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.154003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:34.155210 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:34.155230 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.156008 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.156014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:34.156241 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:34.156881 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:34.156898 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:34.159455 (+ 2557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:34.164075 (+ 4620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:34.164085 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:34.164090 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:34.164729 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:34.165223 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:34.165227 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:34.165230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:34.165405 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:34.166181 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:34.166187 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:34.166333 (+ 146us) client_negotiation.cc:769] Sending connection context
0806 22:40:34.166489 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 22:40:34.166792 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1659,"thread_start_us":1510,"threads_started":1}
I20260806 22:40:34.167465 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:34.148898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39810 (local address 127.15.198.190:40509)
0806 22:40:34.149562 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.149567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.151094 (+ 1527us) server_negotiation.cc:407] Connection header received
0806 22:40:34.151317 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.151325 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.151452 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.151592 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.152842 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.153820 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.155369 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.155803 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.155959 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:34.159677 (+ 3718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:34.159684 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:34.159690 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:34.159738 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:34.163935 (+ 4197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.164928 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.164933 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.164935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.165107 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.165521 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.165525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.165527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.166040 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:34.166241 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:34.166679 (+ 438us) server_negotiation.cc:299] Negotiation successful
0806 22:40:34.166962 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":361,"threads_started":1}
I20260806 22:40:34.169229 15931 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:40:34 dist-test-slave-h3ft krb5kdc[15614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056034, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.188@KRBTEST.COM for kudu/127.15.198.189@KRBTEST.COM
Aug 06 22:40:34 dist-test-slave-h3ft krb5kdc[15614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:40:34.186903 15937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:34.170231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:42677 (local address 127.0.0.1:38198)
0806 22:40:34.170468 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:34.170498 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:34.170620 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:34.171249 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:34.171254 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:34.171810 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:34.172021 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:34.172035 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.173504 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.173509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:34.174970 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:34.174991 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.176008 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.176017 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:34.176197 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:34.176868 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:34.176895 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:34.179360 (+ 2465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:34.184243 (+ 4883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:34.184248 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:34.184251 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:34.184684 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:34.185167 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:34.185170 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:34.185172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:34.185321 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:34.186085 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:34.186088 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:34.186244 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 22:40:34.186360 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:40:34.186614 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 22:40:34.187269 15938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:34.170226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38198 (local address 127.15.198.189:42677)
0806 22:40:34.170828 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.170835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.170858 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:40:34.170935 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.170940 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.171094 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.171190 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.172190 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.173347 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.175160 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.175807 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.175944 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:34.179589 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:34.179598 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:34.179606 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:34.179662 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:34.184086 (+ 4424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.184824 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.184833 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.184839 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.185033 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.185437 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.185443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.185447 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.185951 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:34.186156 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:34.186516 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 22:40:34.186810 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":313,"threads_started":1}
I20260806 22:40:34.187649 15820 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } attempt: 5
I20260806 22:40:34.188999 15931 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:40:34.195437 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:34.168633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39812 (local address 127.15.198.190:40509)
0806 22:40:34.168839 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.168845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.168939 (+ 94us) server_negotiation.cc:407] Connection header received
0806 22:40:34.169123 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.169128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.169246 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.169365 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.170978 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.172289 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.174112 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.174901 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.175114 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:34.187491 (+ 12377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:34.187500 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:34.187506 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:34.187597 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:34.191842 (+ 4245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.192946 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.192954 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.192958 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.193154 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.193690 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.193699 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.193704 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.194076 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:34.194437 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:34.194741 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 22:40:34.195065 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 22:40:34 dist-test-slave-h3ft krb5kdc[15614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056033, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.189@KRBTEST.COM for kudu/127.15.198.188@KRBTEST.COM
Aug 06 22:40:34 dist-test-slave-h3ft krb5kdc[15614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056034, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.188@KRBTEST.COM for kudu/127.15.198.188@KRBTEST.COM
I20260806 22:40:34.206020 15824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:34.188294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46804)
0806 22:40:34.188506 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:34.188527 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:34.188640 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:34.189018 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:34.189024 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:34.189544 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:34.189806 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:34.189819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.191378 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.191386 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:34.192739 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:34.192753 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.193946 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.193953 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:34.194199 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:34.194912 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:34.194943 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:34.198253 (+ 3310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:34.202739 (+ 4486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:34.202746 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:34.202754 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:34.203345 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:34.203746 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:34.203751 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:34.203755 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:34.203943 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:34.204769 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:34.204773 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:34.204970 (+ 197us) client_negotiation.cc:769] Sending connection context
0806 22:40:34.205142 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 22:40:34.205491 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 22:40:34.207337 15935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:34.188392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46804 (local address 127.15.198.188:34055)
0806 22:40:34.188588 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.188595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.188612 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:40:34.188748 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.188754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.188856 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.188956 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.189949 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.191233 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.192906 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.193492 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.193651 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:34.198493 (+ 4842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:34.198500 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:34.198505 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:34.198544 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:34.202588 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.203460 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.203467 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.203471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.203622 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.204062 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.204069 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.204073 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.204538 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:34.204704 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:34.206574 (+ 1870us) server_negotiation.cc:299] Negotiation successful
0806 22:40:34.206875 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 22:40:34.207608 15937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:34.189810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46816)
0806 22:40:34.190295 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:34.190316 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:34.190457 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:34.191447 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:34.191454 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:34.192016 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:34.192209 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:34.192221 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.193782 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.193787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:34.195240 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:34.195261 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.196319 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.196331 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:34.196585 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:34.197208 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:34.197233 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:34.200175 (+ 2942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:34.204894 (+ 4719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:34.204898 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:34.204900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:34.205283 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:34.205658 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:34.205664 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:34.205667 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:34.205808 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:34.206539 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:34.206542 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:34.206657 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 22:40:34.206817 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 22:40:34.207194 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260806 22:40:34.208101 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:34.190053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46816 (local address 127.15.198.188:34055)
0806 22:40:34.191153 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.191158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.191182 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:40:34.191232 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.191238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.191309 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.191439 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.192395 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.193647 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.195427 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.196116 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.196239 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:34.200440 (+ 4201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:34.200449 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:34.200455 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:34.200499 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:34.204543 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.205406 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.205414 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.205417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.205552 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.205960 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.205968 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.205973 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.206319 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:34.206492 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:34.206935 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 22:40:34.207332 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":194,"server-negotiator.queue_time_us":934,"thread_start_us":644,"threads_started":1}
Aug 06 22:40:34 dist-test-slave-h3ft krb5kdc[15614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.189@KRBTEST.COM
I20260806 22:40:34.213794 15820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477: Bootstrap starting.
I20260806 22:40:34.213855 15931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811: Bootstrap starting.
I20260806 22:40:34.215441 15724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } attempt: 5
I20260806 22:40:34.218917 15820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:34.218932 15931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:34.220163 15938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:34.198490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38202 (local address 127.15.198.189:42677)
0806 22:40:34.198703 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.198708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.198783 (+ 75us) server_negotiation.cc:407] Connection header received
0806 22:40:34.198994 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.199026 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.199118 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.199320 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.200880 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.201791 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.203216 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.204036 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.204180 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:34.211971 (+ 7791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:34.211978 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:34.211983 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:34.212025 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:34.215981 (+ 3956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.217024 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.217030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.217033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.217203 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.217711 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.217717 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.217722 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.218164 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:34.218471 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:34.219512 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0806 22:40:34.219821 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 22:40:34.220711 15820 log.cc:824] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:34.221120 15931 log.cc:824] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:34.226811 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:34.228308 15931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811: No bootstrap required, opened a new log
Aug 06 22:40:34 dist-test-slave-h3ft krb5kdc[15614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for kudu/127.15.198.188@KRBTEST.COM
I20260806 22:40:34.232226 15820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477: No bootstrap required, opened a new log
I20260806 22:40:34.230914 15689 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e178c562f72343bc8445cc714b923560"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39812
I20260806 22:40:34.238307 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:34.216299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:34055 (local address 127.0.0.1:46828)
0806 22:40:34.216756 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:34.216780 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:34.216922 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:34.217912 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:34.217918 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:34.218582 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:34.218896 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:34.218916 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.221803 (+ 2887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.221812 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:34.224032 (+ 2220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:34.224057 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.225409 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.225418 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:34.225756 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:34.226708 (+ 952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:34.226744 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:34.230257 (+ 3513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:34.235044 (+ 4787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:34.235051 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:34.235057 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:34.235647 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:34.236169 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:34.236175 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:34.236178 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:34.236370 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:34.237180 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:34.237185 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:34.237389 (+ 204us) client_negotiation.cc:769] Sending connection context
0806 22:40:34.237528 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 22:40:34.237859 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293}
I20260806 22:40:34.239606 15935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:34.216344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46828 (local address 127.15.198.188:34055)
0806 22:40:34.217505 (+ 1161us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.217513 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.217544 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:40:34.217601 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.217607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.217697 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.219114 (+ 1417us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.219268 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.221546 (+ 2278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.224309 (+ 2763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.225190 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.228805 (+ 3615us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:34.230504 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:34.230510 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:34.230515 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:34.230556 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:34.234821 (+ 4265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.235835 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.235842 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.235847 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.236005 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.236528 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.236535 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.236538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.237007 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:34.238739 (+ 1732us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:34.238925 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 22:40:34.239215 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":993}
I20260806 22:40:34.242257 15931 raft_consensus.cc:348] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } }
I20260806 22:40:34.242779 15931 raft_consensus.cc:729] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 935a89a3f5694b9090fc4f66f3142811, State: Initialized, Role: FOLLOWER
I20260806 22:40:34.243427 15931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } }
I20260806 22:40:34.243930 15820 raft_consensus.cc:348] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } }
I20260806 22:40:34.244329 15820 raft_consensus.cc:729] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093a918807294901bc6d8931de2d4477, State: Initialized, Role: FOLLOWER
I20260806 22:40:34.244868 15820 consensus_queue.cc:260] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } }
I20260806 22:40:34.245286 15943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } } }
I20260806 22:40:34.246224 15943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:34.246572 15931 sys_catalog.cc:564] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:34.246565 15724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560: Bootstrap starting.
I20260806 22:40:34.252655 15724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:34.252252 15945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } } }
I20260806 22:40:34.254069 15945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:34.254422 15724 log.cc:824] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:34.254267 15820 sys_catalog.cc:564] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:34.258545 15724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560: No bootstrap required, opened a new log
I20260806 22:40:34.267613 15724 raft_consensus.cc:348] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } }
I20260806 22:40:34.268132 15724 raft_consensus.cc:729] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e178c562f72343bc8445cc714b923560, State: Initialized, Role: FOLLOWER
I20260806 22:40:34.268782 15724 consensus_queue.cc:260] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } }
I20260806 22:40:34.270327 15961 sys_catalog.cc:455] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } } }
I20260806 22:40:34.271136 15961 sys_catalog.cc:458] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:34.272038 15724 sys_catalog.cc:564] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:34.272131 15689 raft_consensus.cc:482] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 22:40:34.272356 15689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:34.276801 15689 raft_consensus.cc:504] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } }
I20260806 22:40:34.280782 15788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e178c562f72343bc8445cc714b923560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "093a918807294901bc6d8931de2d4477"
I20260806 22:40:34.281596 15788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:34.286808 15788 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e178c562f72343bc8445cc714b923560 in term 1.
I20260806 22:40:34.288179 15652 leader_election.cc:302] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093a918807294901bc6d8931de2d4477, e178c562f72343bc8445cc714b923560; no voters:
I20260806 22:40:34.290982 15892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e178c562f72343bc8445cc714b923560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "935a89a3f5694b9090fc4f66f3142811"
I20260806 22:40:34.291692 15892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:34.297103 15892 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e178c562f72343bc8445cc714b923560 in term 1.
I20260806 22:40:34.298331 15689 leader_election.cc:288] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [CANDIDATE]: Term 1 election: Requested vote from peers 093a918807294901bc6d8931de2d4477 (127.15.198.189:42677), 935a89a3f5694b9090fc4f66f3142811 (127.15.198.188:34055)
I20260806 22:40:34.308058 15961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:34.312829 15961 raft_consensus.cc:686] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [term 1 LEADER]: Becoming Leader. State: Replica: e178c562f72343bc8445cc714b923560, State: Running, Role: LEADER
I20260806 22:40:34.313894 15961 consensus_queue.cc:237] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } }
I20260806 22:40:34.322465 15997 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:40:34.331110 16002 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:40:34.337412 16003 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:40:34.349121 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:34.308157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39824 (local address 127.15.198.190:40509)
0806 22:40:34.308392 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.308402 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.324986 (+ 16584us) server_negotiation.cc:407] Connection header received
0806 22:40:34.328625 (+ 3639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.328634 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.328765 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.328861 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.331799 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.333299 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.335631 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.336408 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.336514 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:34.340958 (+ 4444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:34.340966 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:34.340970 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:34.341030 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:34.344960 (+ 3930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.346210 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.346219 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.346225 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.346444 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.346938 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.346943 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.346946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.347404 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:34.347766 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:34.348390 (+ 624us) server_negotiation.cc:299] Negotiation successful
0806 22:40:34.348671 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 22:40:34.352598 15938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:34.308883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38216 (local address 127.15.198.189:42677)
0806 22:40:34.309080 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.309086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.319443 (+ 10357us) server_negotiation.cc:407] Connection header received
0806 22:40:34.322218 (+ 2775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.322226 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.322340 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.322434 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.334619 (+ 12185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.335775 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.339159 (+ 3384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.339826 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.339965 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:34.343832 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:34.343841 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:34.343848 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:34.343926 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:34.348212 (+ 4286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.349371 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.349380 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.349385 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.349602 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.350176 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.350185 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.350189 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.350693 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:34.351056 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:34.351727 (+ 671us) server_negotiation.cc:299] Negotiation successful
0806 22:40:34.352007 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 22:40:34.354792 15990 sys_catalog.cc:455] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e178c562f72343bc8445cc714b923560. Latest consensus state: current_term: 1 leader_uuid: "e178c562f72343bc8445cc714b923560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } } }
I20260806 22:40:34.355424 15990 sys_catalog.cc:458] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:34.356484 16006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260806 22:40:34.356613 15864 tablet.cc:2369] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:34.359824 15935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:34.331494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46836 (local address 127.15.198.188:34055)
0806 22:40:34.331971 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.331977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.339534 (+ 7557us) server_negotiation.cc:407] Connection header received
0806 22:40:34.340051 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.340060 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.340171 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.340263 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.342566 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.343783 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.346119 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:34.347074 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:34.347201 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:34.351329 (+ 4128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:34.351337 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:34.351342 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:34.351392 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:34.355844 (+ 4452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.356955 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.356961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.356965 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.357222 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:34.357822 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:34.357828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:34.357831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:34.358189 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:34.358456 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:34.359084 (+ 628us) server_negotiation.cc:299] Negotiation successful
0806 22:40:34.359380 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307}
I20260806 22:40:34.326532 15837 master_runner.cc:186] Error getting master registration for 127.15.198.190:40509: OK, instance_id { permanent_uuid: "e178c562f72343bc8445cc714b923560" instance_seqno: 1786056032766248 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:40:34.361115 15837 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:40:34.365622 16006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:34.376992 15892 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811 [term 1 FOLLOWER]: Refusing update from remote peer e178c562f72343bc8445cc714b923560: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:34.376988 15788 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477 [term 1 FOLLOWER]: Refusing update from remote peer e178c562f72343bc8445cc714b923560: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:34.378162 15961 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:34.378629 15990 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:34.394059 15943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e178c562f72343bc8445cc714b923560. Latest consensus state: current_term: 1 leader_uuid: "e178c562f72343bc8445cc714b923560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } } }
I20260806 22:40:34.394637 15943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:34.398015 15945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e178c562f72343bc8445cc714b923560. Latest consensus state: current_term: 1 leader_uuid: "e178c562f72343bc8445cc714b923560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } } }
I20260806 22:40:34.398380 15945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:34.401021 16008 mvcc.cc:205] Tried to move back new op lower bound from 7315685516793618432 to 7315685516558999552. Current Snapshot: MvccSnapshot[applied={T|T < 7315685516793618432}]
I20260806 22:40:34.421151 15945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e178c562f72343bc8445cc714b923560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } } }
I20260806 22:40:34.421510 15945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:34.422552 15990 sys_catalog.cc:455] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e178c562f72343bc8445cc714b923560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } } }
I20260806 22:40:34.423053 15990 sys_catalog.cc:458] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:34.423312 15990 sys_catalog.cc:455] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e178c562f72343bc8445cc714b923560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } } }
I20260806 22:40:34.423828 15990 sys_catalog.cc:458] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:34.424984 16006 catalog_manager.cc:1347] Generated new cluster ID: 8467d0219f8e4012b0ad3c19ef7f24f9
I20260806 22:40:34.425124 16006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:34.436285 15943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e178c562f72343bc8445cc714b923560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178c562f72343bc8445cc714b923560" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 40509 } } peers { permanent_uuid: "093a918807294901bc6d8931de2d4477" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 42677 } } peers { permanent_uuid: "935a89a3f5694b9090fc4f66f3142811" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 34055 } } }
I20260806 22:40:34.436614 15943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:34.494318 16006 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:34.496824 16006 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:34.517748 16006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e178c562f72343bc8445cc714b923560: Generated new TSK 0
I20260806 22:40:34.519143 16006 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:40:34.632632 15936 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:34.628731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39840 (local address 127.15.198.190:40509)
0806 22:40:34.629009 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.629017 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.629792 (+ 775us) server_negotiation.cc:407] Connection header received
0806 22:40:34.630171 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.630179 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.630291 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.630396 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.632168 (+ 1772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39840: BlockingRecv error: recv got EOF from 127.0.0.1:39840 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260806 22:40:34.636451 15938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:34.628735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38232 (local address 127.15.198.189:42677)
0806 22:40:34.629063 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.629072 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.633935 (+ 4863us) server_negotiation.cc:407] Connection header received
0806 22:40:34.634286 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.634296 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.634431 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.634544 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.636045 (+ 1501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38232: BlockingRecv error: recv got EOF from 127.0.0.1:38232 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
W20260806 22:40:34.639518 15935 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:34.630017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46840 (local address 127.15.198.188:34055)
0806 22:40:34.630240 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.630247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.637532 (+ 7285us) server_negotiation.cc:407] Connection header received
0806 22:40:34.637838 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.637847 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.637974 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.638069 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.639156 (+ 1087us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46840: BlockingRecv error: recv got EOF from 127.0.0.1:46840 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260806 22:40:34.647372 15938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:34.644694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38244 (local address 127.15.198.189:42677)
0806 22:40:34.644974 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.644981 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.645249 (+ 268us) server_negotiation.cc:407] Connection header received
0806 22:40:34.645576 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.645584 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.645753 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.645880 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.647104 (+ 1224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38244: BlockingRecv error: recv got EOF from 127.0.0.1:38244 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260806 22:40:34.654772 15938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:34.652805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38250 (local address 127.15.198.189:42677)
0806 22:40:34.653008 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.653014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.653167 (+ 153us) server_negotiation.cc:407] Connection header received
0806 22:40:34.653356 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.653361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.653485 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.653607 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.654484 (+ 877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38250: BlockingRecv error: recv got EOF from 127.0.0.1:38250 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 22:40:34.662793 15938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:34.660589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38252 (local address 127.15.198.189:42677)
0806 22:40:34.660805 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.660812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.661212 (+ 400us) server_negotiation.cc:407] Connection header received
0806 22:40:34.661406 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.661412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.661519 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.661637 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.662479 (+ 842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38252: BlockingRecv error: recv got EOF from 127.0.0.1:38252 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 22:40:34.673107 15938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:34.670443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38256 (local address 127.15.198.189:42677)
0806 22:40:34.670791 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.670799 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.671131 (+ 332us) server_negotiation.cc:407] Connection header received
0806 22:40:34.671436 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.671441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.671532 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.671639 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.672730 (+ 1091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38256: BlockingRecv error: recv got EOF from 127.0.0.1:38256 (error 108)
Metrics: {"server-negotiator.queue_time_us":175}
W20260806 22:40:34.688760 15938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:34.686100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38264 (local address 127.15.198.189:42677)
0806 22:40:34.686360 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.686368 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.686872 (+ 504us) server_negotiation.cc:407] Connection header received
0806 22:40:34.687202 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.687210 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.687351 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.687496 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.688436 (+ 940us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38264: BlockingRecv error: recv got EOF from 127.0.0.1:38264 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260806 22:40:34.712502 15938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:34.709819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38278 (local address 127.15.198.189:42677)
0806 22:40:34.710106 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.710114 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.710572 (+ 458us) server_negotiation.cc:407] Connection header received
0806 22:40:34.710824 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.710829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.710928 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.711120 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.712191 (+ 1071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38278: BlockingRecv error: recv got EOF from 127.0.0.1:38278 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
W20260806 22:40:34.752688 15938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:34.750107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38288 (local address 127.15.198.189:42677)
0806 22:40:34.750385 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.750395 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.750868 (+ 473us) server_negotiation.cc:407] Connection header received
0806 22:40:34.751112 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.751119 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.751247 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.751393 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.752331 (+ 938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38288: BlockingRecv error: recv got EOF from 127.0.0.1:38288 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260806 22:40:34.824111 15938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:34.821721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38292 (local address 127.15.198.189:42677)
0806 22:40:34.821957 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.821962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.822147 (+ 185us) server_negotiation.cc:407] Connection header received
0806 22:40:34.822398 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.822403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.822514 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.822616 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.823803 (+ 1187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38292: BlockingRecv error: recv got EOF from 127.0.0.1:38292 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260806 22:40:34.959842 15938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:34.957366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38306 (local address 127.15.198.189:42677)
0806 22:40:34.957694 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:34.957702 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:34.958002 (+ 300us) server_negotiation.cc:407] Connection header received
0806 22:40:34.958311 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:34.958316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:34.958406 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:34.958536 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:34.959487 (+ 951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38306: BlockingRecv error: recv got EOF from 127.0.0.1:38306 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
W20260806 22:40:35.223387 15938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:35.220852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38320 (local address 127.15.198.189:42677)
0806 22:40:35.221226 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:35.221241 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:35.221384 (+ 143us) server_negotiation.cc:407] Connection header received
0806 22:40:35.221667 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:35.221673 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:35.221769 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:35.221890 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:35.223074 (+ 1184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38320: BlockingRecv error: recv got EOF from 127.0.0.1:38320 (error 108)
Metrics: {"server-negotiator.queue_time_us":151}
I20260806 22:40:35.328590 15997 catalog_manager.cc:1259] Loaded cluster ID: 8467d0219f8e4012b0ad3c19ef7f24f9
I20260806 22:40:35.328786 15997 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811: loading cluster ID for follower catalog manager: success
I20260806 22:40:35.332458 15997 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811: acquiring CA information for follower catalog manager: success
I20260806 22:40:35.335855 15997 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 935a89a3f5694b9090fc4f66f3142811: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:40:35.338119 16002 catalog_manager.cc:1259] Loaded cluster ID: 8467d0219f8e4012b0ad3c19ef7f24f9
I20260806 22:40:35.338280 16002 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477: loading cluster ID for follower catalog manager: success
I20260806 22:40:35.341699 16002 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477: acquiring CA information for follower catalog manager: success
I20260806 22:40:35.344733 16002 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 093a918807294901bc6d8931de2d4477: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:40:35.761646 16116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:35.752975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38328 (local address 127.15.198.189:42677)
0806 22:40:35.753957 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:35.753963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:35.753985 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:40:35.754079 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:35.754084 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:35.754388 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:35.754524 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:35.755172 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.756338 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.758454 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.758945 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.759153 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:35.759820 (+ 667us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:35.760584 (+ 764us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:35.760728 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:35.761006 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 22:40:35.761176 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":841,"thread_start_us":655,"threads_started":1}
I20260806 22:40:35.777379 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:35.769918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46850 (local address 127.15.198.188:34055)
0806 22:40:35.770633 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:35.770638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:35.770662 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:40:35.770807 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:35.770812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:35.771080 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:35.771276 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:35.771802 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.772884 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.774515 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.775156 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.775289 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:35.775890 (+ 601us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:35.776616 (+ 726us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:35.776803 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:35.776877 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 22:40:35.776981 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":366,"threads_started":1}
I20260806 22:40:35.794521 16116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:35.785384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38330 (local address 127.15.198.189:42677)
0806 22:40:35.785642 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:35.785651 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:35.787502 (+ 1851us) server_negotiation.cc:407] Connection header received
0806 22:40:35.787838 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:35.787844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:35.787960 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:35.788030 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:35.789259 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.790457 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.792579 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.793043 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.793183 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:35.793628 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:35.793751 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:35.793849 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:35.794001 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 22:40:35.794099 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 22:40:35.795753 16136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:35.785293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39850 (local address 127.15.198.190:40509)
0806 22:40:35.787619 (+ 2326us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:35.787626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:35.787647 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:40:35.787814 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:35.787821 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:35.788121 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:35.788345 (+ 224us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:35.789199 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.790457 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.792706 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.793404 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.794019 (+ 615us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:35.794267 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:35.794874 (+ 607us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:35.795026 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:35.795415 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 22:40:35.795503 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2183,"thread_start_us":2037,"threads_started":1}
I20260806 22:40:35.797232 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:35.786390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46866 (local address 127.15.198.188:34055)
0806 22:40:35.786604 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:35.786610 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:35.788064 (+ 1454us) server_negotiation.cc:407] Connection header received
0806 22:40:35.788267 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:35.788272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:35.788399 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:35.788503 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:35.789353 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.790447 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.794332 (+ 3885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:35.794965 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:35.795121 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:35.795874 (+ 753us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:35.796015 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:35.796080 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:35.796728 (+ 648us) server_negotiation.cc:299] Negotiation successful
0806 22:40:35.796874 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 22:40:35.809157 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 15629
I20260806 22:40:35.810671 15719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:40:35.940759 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 15629
I20260806 22:40:35.979662 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 15735
I20260806 22:40:35.981137 15814 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:40:36.108687 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 15735
I20260806 22:40:36.147656 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 15837
I20260806 22:40:36.149260 15923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:40:36.275688 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 15837
2026-08-06T22:40:36Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5952 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:36 dist-test-slave-h3ft krb5kdc[16149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:36 dist-test-slave-h3ft krb5kdc[16149](info): set up 2 sockets
Aug 06 22:40:36 dist-test-slave-h3ft krb5kdc[16149](info): commencing operation
krb5kdc: starting...
W20260806 22:40:37.685935 16154 mini_kdc.cc:121] Time spent starting KDC: real 1.355s user 0.003s sys 0.070s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/test-user.keytab.
Aug 06 22:40:37 dist-test-slave-h3ft krb5kdc[16149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.198.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
Entry for principal kudu/127.15.198.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab.
I20260806 22:40:38.147472 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:33689
--webserver_interface=127.15.198.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:33689,127.15.198.189:41039,127.15.198.188:37341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:38.607187 16170 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:40:38.607684 16170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:38.607822 16170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:38.607977 16170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:38.616353 16170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:38.616480 16170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:38.616596 16170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:38.616709 16170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:38.616829 16170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:38.628667 16170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:33689,127.15.198.189:41039,127.15.198.188:37341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.190.keytab
--principal=kudu/127.15.198.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.190:33689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.198.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:38.630645 16170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:38.632542 16170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:38.647330 16182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:38.647330 16183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:38.647330 16187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:38.648285 16170 server_base.cc:1034] running on GCE node
I20260806 22:40:38.649305 16170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:38.651207 16170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:38.652487 16170 hybrid_clock.cc:648] HybridClock initialized: now 1786056038652419 us; error 37 us; skew 500 ppm
I20260806 22:40:38.652907 16170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:38 dist-test-slave-h3ft krb5kdc[16149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056038, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:38.656759 16170 init.cc:377] Logged in from keytab as kudu/127.15.198.190@KRBTEST.COM (short username kudu)
I20260806 22:40:38.658603 16170 webserver.cc:460] Webserver started at http://127.15.198.190:34727/ using document root <none> and password file <none>
I20260806 22:40:38.659595 16170 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:38.659739 16170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:38.660137 16170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:38.663913 16170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-0/data/instance:
uuid: "42e370ac803546a0b28437b66e5b41b7"
format_stamp: "Formatted at 2026-08-06 22:40:38 on dist-test-slave-h3ft"
server_key: "927ad76d1ea3a31edd62ee40b5a959dd"
server_key_iv: "42744be49e50b50d5419dc01b7e68180"
server_key_version: "encryptionkey@0"
I20260806 22:40:38.664902 16170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-0/wal/instance:
uuid: "42e370ac803546a0b28437b66e5b41b7"
format_stamp: "Formatted at 2026-08-06 22:40:38 on dist-test-slave-h3ft"
server_key: "927ad76d1ea3a31edd62ee40b5a959dd"
server_key_iv: "42744be49e50b50d5419dc01b7e68180"
server_key_version: "encryptionkey@0"
I20260806 22:40:38.671314 16170 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260806 22:40:38.675729 16194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:38.677441 16170 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 22:40:38.677680 16170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-0/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-0/wal
uuid: "42e370ac803546a0b28437b66e5b41b7"
format_stamp: "Formatted at 2026-08-06 22:40:38 on dist-test-slave-h3ft"
server_key: "927ad76d1ea3a31edd62ee40b5a959dd"
server_key_iv: "42744be49e50b50d5419dc01b7e68180"
server_key_version: "encryptionkey@0"
I20260806 22:40:38.677955 16170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:38.693188 16170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:38.698709 16170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:38.699090 16170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:38.721446 16170 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.190:33689
I20260806 22:40:38.721472 16258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.190:33689 every 8 connection(s)
I20260806 22:40:38.723344 16170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:38.724767 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 16170
I20260806 22:40:38.724994 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-0/wal/instance
I20260806 22:40:38.725543 16154 external_mini_cluster.cc:1428] Setting key b850fd4734898934f748c46a9f8373f7
I20260806 22:40:38.728610 16259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:38.739253 16259 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:40:38 dist-test-slave-h3ft krb5kdc[16149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056038, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.198.189@KRBTEST.COM; defaulting to no policy
I20260806 22:40:38.768045 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:38.744116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33689 (local address 127.0.0.1:47308)
0806 22:40:38.745758 (+ 1642us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:38.745812 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:38.748025 (+ 2213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:38.749088 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:38.749099 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:38.749971 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:38.750396 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:38.750532 (+ 136us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:38.752505 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:38.752511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:38.753763 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:38.753778 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:38.754846 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:38.754853 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:38.755085 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:38.755945 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:38.756020 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:38.758720 (+ 2700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:38.762700 (+ 3980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:38.762715 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:38.762731 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:38.763273 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:38.763724 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:38.763731 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:38.763735 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:38.763907 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:38.764852 (+ 945us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:38.764861 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:38.765130 (+ 269us) client_negotiation.cc:769] Sending connection context
0806 22:40:38.765418 (+ 288us) client_negotiation.cc:240] Negotiation successful
0806 22:40:38.766211 (+ 793us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1399,"mutex_wait_us":228,"spinlock_wait_cycles":63872,"thread_start_us":1165,"threads_started":1}
I20260806 22:40:38.768059 16263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:38.744116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47308 (local address 127.15.198.190:33689)
0806 22:40:38.745263 (+ 1147us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:38.745276 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:38.745968 (+ 692us) server_negotiation.cc:407] Connection header received
0806 22:40:38.748504 (+ 2536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:38.748511 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:38.748821 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:38.749071 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:38.750808 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:38.752322 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:38.753935 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:38.754576 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:38.754804 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:38.758992 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:38.759058 (+ 66us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:38.759088 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:38.759123 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:38.762519 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:38.763420 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:38.763439 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:38.763442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:38.763608 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:38.764044 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:38.764048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:38.764050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:38.764694 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:38.764942 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:38.765578 (+ 636us) server_negotiation.cc:299] Negotiation successful
0806 22:40:38.766083 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":905,"thread_start_us":602,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.189@KRBTEST.COM" created.
I20260806 22:40:38.771919 16259 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:40:38.773206 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:38.772724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:41039 (local address 127.0.0.1:41492)
0806 22:40:38.772927 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:38.773063 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 22:40:38.773494 16197 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.198.189:41039: connect: Connection refused (error 111)
I20260806 22:40:38.775416 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:38.774966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:41039 (local address 127.0.0.1:41500)
0806 22:40:38.775178 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:38.775252 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260806 22:40:38.775610 16259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.189:41039: Network error: Client connection negotiation failed: client connection to 127.15.198.189:41039: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.189.keytab.
Entry for principal kudu/127.15.198.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.189.keytab.
I20260806 22:40:38.822713 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.189:41039
--webserver_interface=127.15.198.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.189.keytab
--principal=kudu/127.15.198.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:33689,127.15.198.189:41039,127.15.198.188:37341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:40:38.825006 16259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } attempt: 1
I20260806 22:40:38.826689 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:38.825933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:41039 (local address 127.0.0.1:41516)
0806 22:40:38.826292 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:38.826453 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260806 22:40:38.828078 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:38.827575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:41039 (local address 127.0.0.1:41518)
0806 22:40:38.827755 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:38.827863 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 22:40:38.828370 16259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.189:41039: Network error: Client connection negotiation failed: client connection to 127.15.198.189:41039: connect: Connection refused (error 111)
I20260806 22:40:38.896678 16259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } attempt: 2
I20260806 22:40:38.898335 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:38.897703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:41039 (local address 127.0.0.1:41524)
0806 22:40:38.898001 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:38.898154 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 22:40:38.899545 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:38.899138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:41039 (local address 127.0.0.1:41536)
0806 22:40:38.899321 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:38.899393 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 22:40:38.899745 16259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.189:41039: Network error: Client connection negotiation failed: client connection to 127.15.198.189:41039: connect: Connection refused (error 111)
I20260806 22:40:38.991080 16259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } attempt: 3
I20260806 22:40:38.992728 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:38.992055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:41039 (local address 127.0.0.1:41548)
0806 22:40:38.992299 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:38.992465 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 22:40:38.994598 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:38.994002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:41039 (local address 127.0.0.1:41562)
0806 22:40:38.994235 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:38.994354 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260806 22:40:38.994902 16259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.189:41039: Network error: Client connection negotiation failed: client connection to 127.15.198.189:41039: connect: Connection refused (error 111)
I20260806 22:40:39.138269 16259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } attempt: 4
I20260806 22:40:39.140132 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.139337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:41039 (local address 127.0.0.1:41576)
0806 22:40:39.139753 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.139901 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202}
I20260806 22:40:39.141438 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.140960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:41039 (local address 127.0.0.1:41580)
0806 22:40:39.141136 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.141250 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.189:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 22:40:39.141768 16259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.189:41039: Network error: Client connection negotiation failed: client connection to 127.15.198.189:41039: connect: Connection refused (error 111)
W20260806 22:40:39.258008 16268 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:40:39.258446 16268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:39.258528 16268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:39.258625 16268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:39.266496 16268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:39.266618 16268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:39.266681 16268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:39.266731 16268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:39.266803 16268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:39.277992 16268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:33689,127.15.198.189:41039,127.15.198.188:37341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.189.keytab
--principal=kudu/127.15.198.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.189:41039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.198.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:39.279853 16268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:39.281543 16268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:39.295837 16273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:39.295732 16278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:39.295931 16274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:39.296934 16268 server_base.cc:1034] running on GCE node
I20260806 22:40:39.297976 16268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:39.299894 16268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:39.301177 16268 hybrid_clock.cc:648] HybridClock initialized: now 1786056039301085 us; error 41 us; skew 500 ppm
I20260806 22:40:39.301622 16268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:39 dist-test-slave-h3ft krb5kdc[16149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056039, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:39.305647 16268 init.cc:377] Logged in from keytab as kudu/127.15.198.189@KRBTEST.COM (short username kudu)
I20260806 22:40:39.307466 16268 webserver.cc:460] Webserver started at http://127.15.198.189:46421/ using document root <none> and password file <none>
I20260806 22:40:39.308332 16268 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:39.308431 16268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:39.308838 16268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:39.312491 16268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-1/data/instance:
uuid: "ae3ca49b1d624448b32185c4e8c2a439"
format_stamp: "Formatted at 2026-08-06 22:40:39 on dist-test-slave-h3ft"
server_key: "232a185c8341063ac6886f88277f40bf"
server_key_iv: "db878e8001b5f95ca37a45bc4e6fd765"
server_key_version: "encryptionkey@0"
I20260806 22:40:39.313405 16268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-1/wal/instance:
uuid: "ae3ca49b1d624448b32185c4e8c2a439"
format_stamp: "Formatted at 2026-08-06 22:40:39 on dist-test-slave-h3ft"
server_key: "232a185c8341063ac6886f88277f40bf"
server_key_iv: "db878e8001b5f95ca37a45bc4e6fd765"
server_key_version: "encryptionkey@0"
I20260806 22:40:39.319543 16268 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 22:40:39.323572 16285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:39.325270 16268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 22:40:39.325510 16268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-1/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-1/wal
uuid: "ae3ca49b1d624448b32185c4e8c2a439"
format_stamp: "Formatted at 2026-08-06 22:40:39 on dist-test-slave-h3ft"
server_key: "232a185c8341063ac6886f88277f40bf"
server_key_iv: "db878e8001b5f95ca37a45bc4e6fd765"
server_key_version: "encryptionkey@0"
I20260806 22:40:39.325809 16268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:39.340770 16268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:39.346513 16268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:39.346904 16268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:39.370589 16268 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.189:41039
I20260806 22:40:39.370609 16359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.189:41039 every 8 connection(s)
I20260806 22:40:39.372542 16268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-1/data/info.pb
I20260806 22:40:39.377779 16360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:39.380234 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 16268
I20260806 22:40:39.380501 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-1/wal/instance
I20260806 22:40:39.381058 16154 external_mini_cluster.cc:1428] Setting key 09003276a96b2c10eca245a20d556a95
I20260806 22:40:39.386180 16360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:40:39 dist-test-slave-h3ft krb5kdc[16149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056039, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.189@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:40:39.417042 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.389549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47314 (local address 127.15.198.190:33689)
0806 22:40:39.390249 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:39.390255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:39.391649 (+ 1394us) server_negotiation.cc:407] Connection header received
0806 22:40:39.394743 (+ 3094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:39.394753 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:39.394909 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:39.395113 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:39.397521 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.398891 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.401481 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.402345 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.402538 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:39.407558 (+ 5020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:39.407568 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:39.407577 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:39.407659 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:39.412523 (+ 4864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:39.413625 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:39.413636 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:39.413643 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:39.413844 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:39.414391 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:39.414399 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:39.414405 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:39.414922 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:39.415134 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:39.416134 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0806 22:40:39.416464 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":427,"threads_started":1}
I20260806 22:40:39.417989 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.390389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33689 (local address 127.0.0.1:47314)
0806 22:40:39.391460 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.391513 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:39.394336 (+ 2823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:39.395370 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:39.395384 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:39.396293 (+ 909us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:39.397197 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:39.397225 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.399115 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.399123 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:39.401194 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:39.401217 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.402640 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.402646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:39.402906 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:39.403930 (+ 1024us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:39.403964 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:39.407210 (+ 3246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:39.412776 (+ 5566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:39.412787 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:39.412809 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:39.413460 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:39.414018 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:39.414022 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:39.414024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:39.414262 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:39.415117 (+ 855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:39.415125 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:39.415605 (+ 480us) client_negotiation.cc:769] Sending connection context
0806 22:40:39.415909 (+ 304us) client_negotiation.cc:240] Negotiation successful
0806 22:40:39.416451 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":484,"threads_started":1}
I20260806 22:40:39.420301 16360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.198.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.198.188@KRBTEST.COM" created.
Aug 06 22:40:39 dist-test-slave-h3ft krb5kdc[16149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056039, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.189@KRBTEST.COM for kudu/127.15.198.189@KRBTEST.COM
I20260806 22:40:39.441102 16259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } attempt: 5
I20260806 22:40:39.442248 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.421140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:41039 (local address 127.0.0.1:41592)
0806 22:40:39.421356 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.421378 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:39.421503 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:39.422783 (+ 1280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:39.422796 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:39.423541 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:39.423803 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:39.423820 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.425407 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.425415 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:39.427080 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:39.427098 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.428109 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.428119 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:39.428356 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:39.429156 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:39.429182 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:39.434680 (+ 5498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:39.438902 (+ 4222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:39.438910 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:39.438917 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:39.439534 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:39.440043 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:39.440050 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:39.440056 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:39.440270 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:39.441010 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:39.441023 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:39.441207 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 22:40:39.441365 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 22:40:39.441729 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 22:40:39.442771 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.421483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41592 (local address 127.15.198.189:41039)
0806 22:40:39.422250 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:39.422256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:39.422288 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:40:39.422339 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:39.422346 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:39.422588 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:39.422730 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:39.424041 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.425242 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.427262 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.427922 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.428044 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:39.435062 (+ 7018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:39.435086 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:39.435107 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:39.435152 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:39.438721 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:39.439708 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:39.439716 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:39.439719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:39.439871 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:39.440386 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:39.440389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:39.440391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:39.440850 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:39.441155 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:39.441712 (+ 557us) server_negotiation.cc:299] Negotiation successful
0806 22:40:39.442049 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":403,"threads_started":1}
I20260806 22:40:39.446878 16360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:40:39.448657 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.447942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37508)
0806 22:40:39.448204 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.448364 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260806 22:40:39.449090 16291 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
I20260806 22:40:39.451429 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.450981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37514)
0806 22:40:39.451187 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.451287 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260806 22:40:39.451802 16360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:37341: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Aug 06 22:40:39 dist-test-slave-h3ft krb5kdc[16149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056038, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for kudu/127.15.198.189@KRBTEST.COM
I20260806 22:40:39.460525 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.442700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41598 (local address 127.15.198.189:41039)
0806 22:40:39.443093 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:39.443099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:39.443119 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:40:39.443171 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:39.443181 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:39.443333 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:39.444484 (+ 1151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:39.444598 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.445813 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.447863 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.448523 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.448735 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:39.453657 (+ 4922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:39.453666 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:39.453674 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:39.453720 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:39.457180 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:39.458168 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:39.458172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:39.458175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:39.458316 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:39.458825 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:39.458829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:39.458831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:39.459350 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:39.459486 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:39.460051 (+ 565us) server_negotiation.cc:299] Negotiation successful
0806 22:40:39.460254 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":624,"threads_started":1}
I20260806 22:40:39.460813 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.442011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:41039 (local address 127.0.0.1:41598)
0806 22:40:39.442516 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.442539 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:39.442671 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:39.443561 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:39.443570 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:39.444110 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:39.444338 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:39.444357 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.445982 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.445990 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:39.447675 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:39.447696 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:39.449351 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:39.449364 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:39.449644 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:39.450437 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:39.450468 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:39.453443 (+ 2975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:39.457345 (+ 3902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:39.457355 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:39.457362 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:39.458013 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:39.458447 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:39.458456 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:39.458462 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:39.458671 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:39.459499 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:39.459507 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:39.459720 (+ 213us) client_negotiation.cc:769] Sending connection context
0806 22:40:39.459900 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 22:40:39.460336 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337}
I20260806 22:40:39.462044 16259 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:40:39.463521 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.463091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37524)
0806 22:40:39.463280 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.463371 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 22:40:39.464639 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.464235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37538)
0806 22:40:39.464441 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.464513 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 22:40:39.464849 16259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:37341: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.198.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.188.keytab.
Entry for principal kudu/127.15.198.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.188.keytab.
W20260806 22:40:39.490304 16154 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.198.188: real 0.109s user 0.001s sys 0.042s
I20260806 22:40:39.491390 16154 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
/tmp/dist-test-taskuRfOoI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.188:37341
--webserver_interface=127.15.198.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.188.keytab
--principal=kudu/127.15.198.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.198.148:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.198.190:33689,127.15.198.189:41039,127.15.198.188:37341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:40:39.501266 16360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } attempt: 1
I20260806 22:40:39.502864 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.502266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37554)
0806 22:40:39.502527 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.502645 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 22:40:39.504130 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.503696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37570)
0806 22:40:39.503902 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.503991 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 22:40:39.504370 16360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:37341: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
I20260806 22:40:39.516119 16259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } attempt: 1
I20260806 22:40:39.517587 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.517009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37576)
0806 22:40:39.517252 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.517372 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 22:40:39.518813 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.518439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37588)
0806 22:40:39.518602 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.518674 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 22:40:39.519114 16259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:37341: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
I20260806 22:40:39.572662 16360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } attempt: 2
I20260806 22:40:39.574483 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.573708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37604)
0806 22:40:39.574013 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.574168 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260806 22:40:39.576100 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.575568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37606)
0806 22:40:39.575766 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.575888 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 22:40:39.576496 16360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:37341: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
I20260806 22:40:39.587375 16259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } attempt: 2
I20260806 22:40:39.588878 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.588271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37608)
0806 22:40:39.588562 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.588676 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260806 22:40:39.590220 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.589721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37610)
0806 22:40:39.589917 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.590049 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 22:40:39.590487 16259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:37341: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
I20260806 22:40:39.667820 16360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } attempt: 3
I20260806 22:40:39.669502 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.668803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37614)
0806 22:40:39.669088 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.669249 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 22:40:39.670902 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.670309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37616)
0806 22:40:39.670606 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.670735 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
W20260806 22:40:39.671160 16360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:37341: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
I20260806 22:40:39.696890 16259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } attempt: 3
I20260806 22:40:39.698815 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.697897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37630)
0806 22:40:39.698292 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.698463 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260806 22:40:39.700255 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.699755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37632)
0806 22:40:39.699979 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.700100 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260806 22:40:39.700512 16259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:37341: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
I20260806 22:40:39.814424 16360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } attempt: 4
I20260806 22:40:39.816339 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.815525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37646)
0806 22:40:39.815838 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.816031 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260806 22:40:39.817824 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.817277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37660)
0806 22:40:39.817469 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.817601 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 22:40:39.818173 16360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:37341: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
I20260806 22:40:39.877799 16259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } attempt: 4
I20260806 22:40:39.879611 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.878874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37674)
0806 22:40:39.879224 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.879367 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260806 22:40:39.881004 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:39.880490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37682)
0806 22:40:39.880676 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:39.880796 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 22:40:39.881296 16259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.198.188:37341: Network error: Client connection negotiation failed: client connection to 127.15.198.188:37341: connect: Connection refused (error 111)
W20260806 22:40:39.933911 16378 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:40:39.934381 16378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:39.934515 16378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:39.934628 16378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:39.942955 16378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:39.943114 16378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:39.943223 16378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:39.943327 16378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:39.943440 16378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:39.955197 16378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.198.148:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.198.190:33689,127.15.198.189:41039,127.15.198.188:37341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/krb5kdc/kudu_127.15.198.188.keytab
--principal=kudu/127.15.198.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.198.188:37341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.198.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 21:40:31 UTC on ccycloud.quasar-sprmbg.root.comops.site
build id 4571537
ASAN enabled
I20260806 22:40:39.957031 16378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:39.958796 16378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:39.973291 16386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:39.973255 16388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:39.973326 16391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:39.974457 16378 server_base.cc:1034] running on GCE node
I20260806 22:40:39.975581 16378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:39.977523 16378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:39.978796 16378 hybrid_clock.cc:648] HybridClock initialized: now 1786056039978712 us; error 60 us; skew 500 ppm
I20260806 22:40:39.979254 16378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 22:40:39 dist-test-slave-h3ft krb5kdc[16149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056039, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:39.983202 16378 init.cc:377] Logged in from keytab as kudu/127.15.198.188@KRBTEST.COM (short username kudu)
I20260806 22:40:39.984877 16378 webserver.cc:460] Webserver started at http://127.15.198.188:34403/ using document root <none> and password file <none>
I20260806 22:40:39.985765 16378 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:39.985903 16378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:39.986279 16378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:39.989907 16378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-2/data/instance:
uuid: "265f37875e2b4377b9c44a5ac47d2014"
format_stamp: "Formatted at 2026-08-06 22:40:39 on dist-test-slave-h3ft"
server_key: "ba992562a2570de30820e3d5ff8bba2d"
server_key_iv: "2953c8b38c2c2306f9fc60ea173e077d"
server_key_version: "encryptionkey@0"
I20260806 22:40:39.990831 16378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-2/wal/instance:
uuid: "265f37875e2b4377b9c44a5ac47d2014"
format_stamp: "Formatted at 2026-08-06 22:40:39 on dist-test-slave-h3ft"
server_key: "ba992562a2570de30820e3d5ff8bba2d"
server_key_iv: "2953c8b38c2c2306f9fc60ea173e077d"
server_key_version: "encryptionkey@0"
I20260806 22:40:39.997321 16378 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 22:40:40.001644 16400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:40.003710 16378 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 22:40:40.004056 16378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-2/data,/tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-2/wal
uuid: "265f37875e2b4377b9c44a5ac47d2014"
format_stamp: "Formatted at 2026-08-06 22:40:39 on dist-test-slave-h3ft"
server_key: "ba992562a2570de30820e3d5ff8bba2d"
server_key_iv: "2953c8b38c2c2306f9fc60ea173e077d"
server_key_version: "encryptionkey@0"
I20260806 22:40:40.004408 16378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:40.023772 16378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:40.029353 16378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:40.029656 16378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:40.054512 16466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.198.188:37341 every 8 connection(s)
I20260806 22:40:40.054503 16378 rpc_server.cc:307] RPC server started. Bound to: 127.15.198.188:37341
I20260806 22:40:40.056587 16378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-2/data/info.pb
I20260806 22:40:40.061105 16154 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu as pid 16378
I20260806 22:40:40.061381 16154 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuRfOoI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055595907839-16154-0/minicluster-data/master-2/wal/instance
I20260806 22:40:40.062081 16154 external_mini_cluster.cc:1428] Setting key 90b30f48887d27c9220ac9ffd5a19007
I20260806 22:40:40.062216 16469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:40.074273 16469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:40:40 dist-test-slave-h3ft krb5kdc[16149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.188@KRBTEST.COM
Aug 06 22:40:40 dist-test-slave-h3ft krb5kdc[16149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056039, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.188@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
I20260806 22:40:40.093442 16474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.064524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37694 (local address 127.15.198.188:37341)
0806 22:40:40.065504 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.065519 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.065573 (+ 54us) server_negotiation.cc:407] Connection header received
0806 22:40:40.068085 (+ 2512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.068118 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.068703 (+ 585us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.069172 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.071507 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.073064 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.075084 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.076041 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.076218 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.082309 (+ 6091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:40.082323 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:40.082340 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:40.082397 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:40.086817 (+ 4420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.088127 (+ 1310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.088135 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.088144 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.088347 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.088979 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.088986 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.088989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.089912 (+ 923us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:40.090191 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:40.090855 (+ 664us) server_negotiation.cc:299] Negotiation successful
0806 22:40:40.091542 (+ 687us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":464,"threads_started":1}
I20260806 22:40:40.098680 16476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.078409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47318 (local address 127.15.198.190:33689)
0806 22:40:40.079190 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.079198 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.079860 (+ 662us) server_negotiation.cc:407] Connection header received
0806 22:40:40.080093 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.080100 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.080229 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.080381 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.082318 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.083613 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.085335 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.085982 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.086153 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.090786 (+ 4633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:40.090795 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:40.090800 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:40.090856 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:40.095062 (+ 4206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.096146 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.096151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.096154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.096313 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.096811 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.096818 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.096823 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.097245 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:40.097407 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:40.097971 (+ 564us) server_negotiation.cc:299] Negotiation successful
0806 22:40:40.098250 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":353,"threads_started":1}
I20260806 22:40:40.098949 16478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.078425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.190:33689 (local address 127.0.0.1:47318)
0806 22:40:40.079669 (+ 1244us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:40.079724 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:40.079923 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:40.080471 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:40.080477 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:40.081588 (+ 1111us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:40.082136 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:40.082151 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.083820 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.083824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:40.085130 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:40.085147 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.086201 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.086206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:40.086348 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.087169 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:40.087201 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:40.090521 (+ 3320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:40.095240 (+ 4719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:40.095251 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:40.095257 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:40.095967 (+ 710us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:40.096453 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:40.096467 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:40.096470 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:40.096672 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:40.097388 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:40.097397 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:40.097610 (+ 213us) client_negotiation.cc:769] Sending connection context
0806 22:40:40.097842 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 22:40:40.098304 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":939,"thread_start_us":738,"threads_started":1}
I20260806 22:40:40.101094 16469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:40:40 dist-test-slave-h3ft krb5kdc[16149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.190@KRBTEST.COM
Aug 06 22:40:40 dist-test-slave-h3ft krb5kdc[16149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056039, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.188@KRBTEST.COM for kudu/127.15.198.189@KRBTEST.COM
I20260806 22:40:40.117354 16478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.101899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.189:41039 (local address 127.0.0.1:41602)
0806 22:40:40.102246 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:40.102269 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:40.102376 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:40.103153 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:40.103159 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:40.103610 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:40.103782 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:40.103792 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.104881 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.104888 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:40.105819 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:40.105831 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.106686 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.106692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:40.106903 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.107618 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:40.107635 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:40.110135 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:40.114460 (+ 4325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:40.114464 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:40.114467 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:40.114882 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:40.115338 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:40.115344 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:40.115348 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:40.115525 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:40.116259 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:40.116267 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:40.116473 (+ 206us) client_negotiation.cc:769] Sending connection context
0806 22:40:40.116642 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 22:40:40.116959 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224}
I20260806 22:40:40.117419 16360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } attempt: 5
I20260806 22:40:40.118402 16476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.097903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47334 (local address 127.15.198.190:33689)
0806 22:40:40.099083 (+ 1180us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.099089 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.099108 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:40:40.099168 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.099173 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.099266 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.099400 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.101135 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.102186 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.103740 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.104200 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.104380 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.109323 (+ 4943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:40.109329 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:40.109332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:40.109371 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:40.113336 (+ 3965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.114152 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.114160 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.114165 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.114330 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.115429 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.115434 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.115436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.115764 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:40.116095 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:40.117149 (+ 1054us) server_negotiation.cc:299] Negotiation successful
0806 22:40:40.118019 (+ 870us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1046,"thread_start_us":1716,"threads_started":1}
I20260806 22:40:40.118445 16483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.102114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41602 (local address 127.15.198.189:41039)
0806 22:40:40.102795 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.102800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.102816 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:40:40.102876 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.102886 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.102962 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.103137 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.104004 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.104738 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.105982 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.106492 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.106644 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.110419 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:40.110426 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:40.110430 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:40.110481 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:40.114321 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.115075 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.115079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.115082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.115218 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.115637 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.115644 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.115647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.115949 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:40.116123 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:40.117700 (+ 1577us) server_negotiation.cc:299] Negotiation successful
0806 22:40:40.117970 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":452,"threads_started":1}
I20260806 22:40:40.119688 16469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:40:40 dist-test-slave-h3ft krb5kdc[16149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056039, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.189@KRBTEST.COM for kudu/127.15.198.188@KRBTEST.COM
Aug 06 22:40:40 dist-test-slave-h3ft krb5kdc[16149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056039, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.188@KRBTEST.COM for kudu/127.15.198.188@KRBTEST.COM
Aug 06 22:40:40 dist-test-slave-h3ft krb5kdc[16149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.198.189@KRBTEST.COM
I20260806 22:40:40.137310 16474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.119339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37696 (local address 127.15.198.188:37341)
0806 22:40:40.119544 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.119550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.119570 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:40:40.119623 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.119629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.119728 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.120564 (+ 836us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.120915 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.122122 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.124014 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.124736 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.124870 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.128947 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:40.128953 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:40.128958 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:40.128997 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:40.133659 (+ 4662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.134666 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.134674 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.134681 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.134832 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.135415 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.135421 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.135428 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.135890 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:40.136048 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:40.136544 (+ 496us) server_negotiation.cc:299] Negotiation successful
0806 22:40:40.136858 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":61}
I20260806 22:40:40.137348 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.118843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37696)
0806 22:40:40.119089 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:40.119109 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:40.119223 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:40.119933 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:40.119939 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:40.120442 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:40.120714 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:40.120728 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.122289 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.122295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:40.123841 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:40.123860 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.124889 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.124896 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:40.125176 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.125856 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:40.125876 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:40.128699 (+ 2823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:40.133849 (+ 5150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:40.133856 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:40.133863 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:40.134500 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:40.135091 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:40.135096 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:40.135100 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:40.135278 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:40.136025 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:40.136032 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:40.136236 (+ 204us) client_negotiation.cc:769] Sending connection context
0806 22:40:40.136407 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 22:40:40.136842 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260806 22:40:40.138224 16486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.120908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37698 (local address 127.15.198.188:37341)
0806 22:40:40.122009 (+ 1101us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.122015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.122038 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:40:40.122092 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.122096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.122165 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.123084 (+ 919us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.123354 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.124438 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.126125 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.126768 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.127566 (+ 798us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.131289 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:40.131294 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:40.131297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:40.131326 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:40.134797 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.135483 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.135487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.135489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.135698 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.136119 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.136123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.136126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.136479 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:40.137407 (+ 928us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:40.137561 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 22:40:40.137757 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":941,"spinlock_wait_cycles":68352,"thread_start_us":664,"threads_started":1}
I20260806 22:40:40.138726 16478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.120795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37698)
0806 22:40:40.121133 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:40.121158 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:40.121286 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:40.122318 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:40.122322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:40.122887 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:40.123206 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:40.123223 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.124560 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.124565 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:40.125966 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:40.125981 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.126927 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.126933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:40.127222 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.127948 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:40.127973 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:40.131050 (+ 3077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:40.134925 (+ 3875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:40.134931 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:40.134934 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:40.135370 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:40.135811 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:40.135817 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:40.135821 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:40.135997 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:40.136593 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:40.136598 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:40.136751 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 22:40:40.136902 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 22:40:40.137237 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260806 22:40:40.141850 16483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.122107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41612 (local address 127.15.198.189:41039)
0806 22:40:40.122299 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.122304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.122321 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:40:40.122512 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.122519 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.122615 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.122729 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.124452 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.125602 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.127529 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.128128 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.128272 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.134116 (+ 5844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:40.134122 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:40.134127 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:40.134169 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:40.138487 (+ 4318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.139462 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.139468 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.139471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.139668 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.140161 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.140167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.140170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.140588 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:40.141013 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:40.141229 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 22:40:40.141488 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:40:40.145146 16360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439: Bootstrap starting.
I20260806 22:40:40.145164 16469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014: Bootstrap starting.
I20260806 22:40:40.147920 16154 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:40.149462 16360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:40.149478 16469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:40.150723 16360 log.cc:824] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:40.150789 16469 log.cc:824] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:40.154428 16360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439: No bootstrap required, opened a new log
I20260806 22:40:40.155242 16469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014: No bootstrap required, opened a new log
I20260806 22:40:40.151593 16232 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "42e370ac803546a0b28437b66e5b41b7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47334
I20260806 22:40:40.158578 16259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } attempt: 5
I20260806 22:40:40.166788 16360 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } }
I20260806 22:40:40.166791 16469 raft_consensus.cc:348] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } }
I20260806 22:40:40.167316 16360 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae3ca49b1d624448b32185c4e8c2a439, State: Initialized, Role: FOLLOWER
I20260806 22:40:40.167317 16469 raft_consensus.cc:729] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 265f37875e2b4377b9c44a5ac47d2014, State: Initialized, Role: FOLLOWER
I20260806 22:40:40.167946 16469 consensus_queue.cc:260] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } }
I20260806 22:40:40.167958 16360 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } }
Aug 06 22:40:40 dist-test-slave-h3ft krb5kdc[16149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056038, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.198.190@KRBTEST.COM for kudu/127.15.198.188@KRBTEST.COM
I20260806 22:40:40.169836 16490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } } }
I20260806 22:40:40.171066 16469 sys_catalog.cc:564] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:40.171319 16490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:40.172714 16489 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } } }
I20260806 22:40:40.173652 16489 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:40.174127 16360 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:40.180883 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.159323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.198.188:37341 (local address 127.0.0.1:37702)
0806 22:40:40.159647 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:40.159671 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:40.159806 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:40.160222 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:40.160228 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:40.160866 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:40.161136 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:40.161152 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.162759 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.162764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:40.164378 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:40.164395 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.165720 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.165726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:40.165943 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.166817 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:40.166848 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:40.170262 (+ 3414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:40.175777 (+ 5515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:40.175784 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:40.175789 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:40.176981 (+ 1192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:40.177702 (+ 721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:40.177710 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:40.177717 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:40.178080 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:40.179225 (+ 1145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:40.179234 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:40.179512 (+ 278us) client_negotiation.cc:769] Sending connection context
0806 22:40:40.179702 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 22:40:40.180201 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260806 22:40:40.182457 16486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.159559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37702 (local address 127.15.198.188:37341)
0806 22:40:40.159786 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.159796 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.159837 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:40:40.159942 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.159951 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.160051 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.160161 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.161330 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.162568 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.164558 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.165333 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.165465 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.170529 (+ 5064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:40.170536 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:40.170542 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:40.170585 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:40.175533 (+ 4948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.177195 (+ 1662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.177204 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.177211 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.177481 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.178267 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.178273 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.178277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.178873 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:40.179131 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:40.181616 (+ 2485us) server_negotiation.cc:299] Negotiation successful
0806 22:40:40.181970 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:40:40.192193 16259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7: Bootstrap starting.
I20260806 22:40:40.199729 16259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:40.201493 16259 log.cc:824] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:40.210666 16259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7: No bootstrap required, opened a new log
I20260806 22:40:40.199472 16378 master_runner.cc:186] Error getting master registration for 127.15.198.190:33689: OK, instance_id { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" instance_seqno: 1786056038715093 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260806 22:40:40.213136 16526 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:40:40.213442 16526 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:40:40.213454 16378 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:40:40.221511 16259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } }
I20260806 22:40:40.222205 16259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42e370ac803546a0b28437b66e5b41b7, State: Initialized, Role: FOLLOWER
I20260806 22:40:40.222877 16259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } }
I20260806 22:40:40.216550 16268 master_runner.cc:186] Error getting master registration for 127.15.198.190:33689: OK, instance_id { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" instance_seqno: 1786056038715093 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:40:40.234409 16268 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 22:40:40.234858 16532 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:40:40.235028 16532 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:40:40.235878 16534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } } }
I20260806 22:40:40.236435 16534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:40.237624 16259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:40.237715 16232 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 22:40:40.237957 16232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:40.242935 16232 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } }
I20260806 22:40:40.245496 16232 leader_election.cc:288] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [CANDIDATE]: Term 1 election: Requested vote from peers ae3ca49b1d624448b32185c4e8c2a439 (127.15.198.189:41039), 265f37875e2b4377b9c44a5ac47d2014 (127.15.198.188:37341)
I20260806 22:40:40.246562 16328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "42e370ac803546a0b28437b66e5b41b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ae3ca49b1d624448b32185c4e8c2a439"
I20260806 22:40:40.246575 16432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "42e370ac803546a0b28437b66e5b41b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "265f37875e2b4377b9c44a5ac47d2014"
I20260806 22:40:40.247212 16328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:40.247308 16432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:40.251365 16432 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42e370ac803546a0b28437b66e5b41b7 in term 1.
I20260806 22:40:40.252280 16199 leader_election.cc:302] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 265f37875e2b4377b9c44a5ac47d2014, 42e370ac803546a0b28437b66e5b41b7; no voters:
I20260806 22:40:40.252919 16534 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:40.254261 16328 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42e370ac803546a0b28437b66e5b41b7 in term 1.
I20260806 22:40:40.268572 16534 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [term 1 LEADER]: Becoming Leader. State: Replica: 42e370ac803546a0b28437b66e5b41b7, State: Running, Role: LEADER
I20260806 22:40:40.269522 16534 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } }
W20260806 22:40:40.282884 16406 tablet.cc:2369] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:40.293642 16538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42e370ac803546a0b28437b66e5b41b7. Latest consensus state: current_term: 1 leader_uuid: "42e370ac803546a0b28437b66e5b41b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } } }
I20260806 22:40:40.294205 16538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:40.315824 16483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.270728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41616 (local address 127.15.198.189:41039)
0806 22:40:40.270923 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.270930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.296058 (+ 25128us) server_negotiation.cc:407] Connection header received
0806 22:40:40.296294 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.296303 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.296422 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.296568 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.299046 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.300291 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.302435 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.303271 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.303402 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.307426 (+ 4024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:40.307436 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:40.307441 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:40.307511 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:40.311722 (+ 4211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.312699 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.312706 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.312711 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.312936 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.313479 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.313485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.313502 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.313974 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:40.314316 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:40.315026 (+ 710us) server_negotiation.cc:299] Negotiation successful
0806 22:40:40.315342 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 22:40:40.315872 16557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:40.322962 16476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.265647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47346 (local address 127.15.198.190:33689)
0806 22:40:40.265875 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.265884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.270385 (+ 4501us) server_negotiation.cc:407] Connection header received
0806 22:40:40.272110 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.272121 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.272254 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.272381 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.283525 (+ 11144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.284949 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.299576 (+ 14627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.300317 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.300448 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.310626 (+ 10178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:40.310635 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:40.310642 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:40.310705 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:40.314938 (+ 4233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.319815 (+ 4877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.319826 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.319832 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.320104 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.320633 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.320642 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.320645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.321154 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:40.321518 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:40.322135 (+ 617us) server_negotiation.cc:299] Negotiation successful
0806 22:40:40.322516 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:40:40.323316 16486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:40.265116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37714 (local address 127.15.198.188:37341)
0806 22:40:40.265380 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.265388 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.271507 (+ 6119us) server_negotiation.cc:407] Connection header received
0806 22:40:40.274857 (+ 3350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.274869 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.275051 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.275178 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.279389 (+ 4211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.281099 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.292301 (+ 11202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:40.293448 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:40.293622 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:40.304426 (+ 10804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:40.304442 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:40.304453 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:40.304516 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:40.309481 (+ 4965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.320277 (+ 10796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.320284 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.320288 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.320520 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:40.321073 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:40.321079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:40.321083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:40.321508 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:40.321862 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:40.322477 (+ 615us) server_negotiation.cc:299] Negotiation successful
0806 22:40:40.322794 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 22:40:40.337335 16557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260806 22:40:40.350639 16293 tablet.cc:2369] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:40.351506 16561 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:40:40.374809 16432 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014 [term 1 FOLLOWER]: Refusing update from remote peer 42e370ac803546a0b28437b66e5b41b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:40.376650 16534 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:40.378476 16328 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439 [term 1 FOLLOWER]: Refusing update from remote peer 42e370ac803546a0b28437b66e5b41b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:40.379978 16538 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:40.418839 16489 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42e370ac803546a0b28437b66e5b41b7. Latest consensus state: current_term: 1 leader_uuid: "42e370ac803546a0b28437b66e5b41b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } } }
I20260806 22:40:40.419471 16489 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:40.428012 16490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42e370ac803546a0b28437b66e5b41b7. Latest consensus state: current_term: 1 leader_uuid: "42e370ac803546a0b28437b66e5b41b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } } }
I20260806 22:40:40.428465 16490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:40.438215 16489 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "42e370ac803546a0b28437b66e5b41b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } } }
I20260806 22:40:40.438623 16489 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:40.442293 16557 catalog_manager.cc:1347] Generated new cluster ID: 0a3ba963b46c40e9a25fa13730b42548
I20260806 22:40:40.442430 16557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:40.447212 16534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "42e370ac803546a0b28437b66e5b41b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } } }
I20260806 22:40:40.447616 16534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:40.448055 16534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "42e370ac803546a0b28437b66e5b41b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } } }
I20260806 22:40:40.448336 16534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:40.452257 16490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "42e370ac803546a0b28437b66e5b41b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42e370ac803546a0b28437b66e5b41b7" member_type: VOTER last_known_addr { host: "127.15.198.190" port: 33689 } } peers { permanent_uuid: "ae3ca49b1d624448b32185c4e8c2a439" member_type: VOTER last_known_addr { host: "127.15.198.189" port: 41039 } } peers { permanent_uuid: "265f37875e2b4377b9c44a5ac47d2014" member_type: VOTER last_known_addr { host: "127.15.198.188" port: 37341 } } }
I20260806 22:40:40.452603 16490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:40:40.475463 16557 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:40.477928 16557 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:40.507301 16557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42e370ac803546a0b28437b66e5b41b7: Generated new TSK 0
I20260806 22:40:40.509084 16557 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:40:40.635689 16154 mini_kdc.cc:348] Time spent kdestroy: real 0.112s user 0.001s sys 0.043s
W20260806 22:40:40.689920 16476 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:40.672068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47362 (local address 127.15.198.190:33689)
0806 22:40:40.672356 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.672365 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.684511 (+ 12146us) server_negotiation.cc:407] Connection header received
0806 22:40:40.687711 (+ 3200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.687721 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.687840 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.687929 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.689502 (+ 1573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47362: BlockingRecv error: recv got EOF from 127.0.0.1:47362 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 22:40:40.693924 16486 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:40.680558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37726 (local address 127.15.198.188:37341)
0806 22:40:40.680816 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.680824 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.691279 (+ 10455us) server_negotiation.cc:407] Connection header received
0806 22:40:40.691927 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.691934 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.692047 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.692145 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.693599 (+ 1454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37726: BlockingRecv error: recv got EOF from 127.0.0.1:37726 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 22:40:40.695981 16483 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:40.685220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41620 (local address 127.15.198.189:41039)
0806 22:40:40.685468 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.685475 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.693757 (+ 8282us) server_negotiation.cc:407] Connection header received
0806 22:40:40.694089 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.694097 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.694217 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.694308 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.695669 (+ 1361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41620: BlockingRecv error: recv got EOF from 127.0.0.1:41620 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 22:40:40.764493 16483 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:40.748030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41636 (local address 127.15.198.189:41039)
0806 22:40:40.748317 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.748324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.761483 (+ 13159us) server_negotiation.cc:407] Connection header received
0806 22:40:40.762266 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.762275 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.762500 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.762594 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.764152 (+ 1558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41636: BlockingRecv error: recv got EOF from 127.0.0.1:41636 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260806 22:40:40.826337 16483 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:40.810631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41646 (local address 127.15.198.189:41039)
0806 22:40:40.810943 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.810951 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.823862 (+ 12911us) server_negotiation.cc:407] Connection header received
0806 22:40:40.824313 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.824325 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.824473 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.824596 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.826002 (+ 1406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41646: BlockingRecv error: recv got EOF from 127.0.0.1:41646 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260806 22:40:40.881134 16483 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:40.867306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41658 (local address 127.15.198.189:41039)
0806 22:40:40.867575 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.867582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.878337 (+ 10755us) server_negotiation.cc:407] Connection header received
0806 22:40:40.879135 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.879144 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.879280 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.879384 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.880792 (+ 1408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41658: BlockingRecv error: recv got EOF from 127.0.0.1:41658 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260806 22:40:40.936125 16483 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:40.922004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41672 (local address 127.15.198.189:41039)
0806 22:40:40.922393 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.922402 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.933353 (+ 10951us) server_negotiation.cc:407] Connection header received
0806 22:40:40.934056 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.934063 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.934193 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.934290 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.935770 (+ 1480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41672: BlockingRecv error: recv got EOF from 127.0.0.1:41672 (error 108)
Metrics: {"server-negotiator.queue_time_us":165}
W20260806 22:40:40.984334 16483 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:40.981380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41688 (local address 127.15.198.189:41039)
0806 22:40:40.981663 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:40.981671 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:40.982298 (+ 627us) server_negotiation.cc:407] Connection header received
0806 22:40:40.982547 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:40.982553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:40.982698 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:40.982816 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:40.983971 (+ 1155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41688: BlockingRecv error: recv got EOF from 127.0.0.1:41688 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260806 22:40:41.007639 16483 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:41.005209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41702 (local address 127.15.198.189:41039)
0806 22:40:41.005486 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:41.005493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:41.005808 (+ 315us) server_negotiation.cc:407] Connection header received
0806 22:40:41.006040 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:41.006046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:41.006134 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:41.006231 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:41.007342 (+ 1111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41702: BlockingRecv error: recv got EOF from 127.0.0.1:41702 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260806 22:40:41.045903 16483 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:41.043604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41712 (local address 127.15.198.189:41039)
0806 22:40:41.043840 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:41.043845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:41.044242 (+ 397us) server_negotiation.cc:407] Connection header received
0806 22:40:41.044451 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:41.044456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:41.044555 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:41.044668 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:41.045638 (+ 970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41712: BlockingRecv error: recv got EOF from 127.0.0.1:41712 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260806 22:40:41.116988 16483 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:41.114645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41722 (local address 127.15.198.189:41039)
0806 22:40:41.114906 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:41.114912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:41.115392 (+ 480us) server_negotiation.cc:407] Connection header received
0806 22:40:41.115598 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:41.115603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:41.115706 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:41.115829 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:41.116749 (+ 920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41722: BlockingRecv error: recv got EOF from 127.0.0.1:41722 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 22:40:41.216372 16526 catalog_manager.cc:1259] Loaded cluster ID: 0a3ba963b46c40e9a25fa13730b42548
I20260806 22:40:41.216547 16526 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014: loading cluster ID for follower catalog manager: success
I20260806 22:40:41.219635 16526 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014: acquiring CA information for follower catalog manager: success
I20260806 22:40:41.222100 16526 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 265f37875e2b4377b9c44a5ac47d2014: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:40:41.237777 16532 catalog_manager.cc:1259] Loaded cluster ID: 0a3ba963b46c40e9a25fa13730b42548
I20260806 22:40:41.237959 16532 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439: loading cluster ID for follower catalog manager: success
I20260806 22:40:41.241114 16532 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439: acquiring CA information for follower catalog manager: success
I20260806 22:40:41.243414 16532 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ae3ca49b1d624448b32185c4e8c2a439: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:40:41.256749 16483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:41.249540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41736 (local address 127.15.198.189:41039)
0806 22:40:41.249744 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:41.249749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:41.250199 (+ 450us) server_negotiation.cc:407] Connection header received
0806 22:40:41.250403 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:41.250408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:41.250597 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:41.250736 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:41.251368 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.252221 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.253696 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.254138 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.254248 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:41.255101 (+ 853us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:41.255925 (+ 824us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:41.256084 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:41.256271 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 22:40:41.256377 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:40:41.270807 16658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:41.263908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37728 (local address 127.15.198.188:37341)
0806 22:40:41.264520 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:41.264526 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:41.264548 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:40:41.264760 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:41.264765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:41.265010 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:41.265195 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:41.265644 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.266482 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.268063 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.268513 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.268616 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:41.269321 (+ 705us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:41.270095 (+ 774us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:41.270249 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:41.270398 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 22:40:41.270486 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":331,"threads_started":1}
I20260806 22:40:41.288810 16658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:41.278932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37742 (local address 127.15.198.188:37341)
0806 22:40:41.279232 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:41.279239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:41.281010 (+ 1771us) server_negotiation.cc:407] Connection header received
0806 22:40:41.281355 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:41.281362 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:41.281488 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:41.281567 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:41.283266 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.284465 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.286361 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.287114 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.287257 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:41.287837 (+ 580us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:41.288023 (+ 186us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:41.288129 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:41.288322 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 22:40:41.288424 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 22:40:41.289085 16483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:41.280425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41748 (local address 127.15.198.189:41039)
0806 22:40:41.280714 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:41.280724 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:41.281294 (+ 570us) server_negotiation.cc:407] Connection header received
0806 22:40:41.281594 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:41.281602 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:41.281712 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:41.281791 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:41.282818 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.284225 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.286611 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.287425 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.287586 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:41.288066 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:41.288291 (+ 225us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:41.288385 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:41.288614 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 22:40:41.288739 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 22:40:41.289551 16667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:41.278694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47372 (local address 127.15.198.190:33689)
0806 22:40:41.279567 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:41.279578 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:41.280803 (+ 1225us) server_negotiation.cc:407] Connection header received
0806 22:40:41.281001 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:41.281012 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:41.281345 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:41.281592 (+ 247us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:41.282498 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.283713 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.286324 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:41.286925 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:41.287131 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:41.288004 (+ 873us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:41.288637 (+ 633us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:41.288779 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:41.289063 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 22:40:41.289170 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":449,"threads_started":1}
I20260806 22:40:41.303236 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 16170
I20260806 22:40:41.304447 16253 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:40:41.433816 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 16170
I20260806 22:40:41.473356 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 16268
I20260806 22:40:41.475111 16355 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:40:41.603243 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 16268
I20260806 22:40:41.636296 16154 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu pid 16378
I20260806 22:40:41.637825 16462 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 22:40:41.782747 16154 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuRfOoI/build/asan/bin/kudu with pid 16378
2026-08-06T22:40:41Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5510 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (11463 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (445801 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST